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PREFACE 


India is regarded as under performer with regard to economic targets and human 
development targets. It could be due to wrong policies or due to poor implementation of 
right policies. It could also be due to wrong policy making processes. The literature on 
the policy making process is far less than the literature on policy issues and on its 
analysis. 

More than twelve thousand officials without any professional knowledge of 
policy components and formulation processes formulate, implement and evaluate policies 
at central, state, district and institutional levels on wide arena of subjects. 

The NUEPA, only university in education in India has taken the onus of 
producing professionals for educational Planning and policy making. The learning 
material has been designed as introductory course for the Educational policy. It will 
enable the reader to understand history of educational policies in India and role of 
government, policy formulation processes to be adopted and critically evaluate policy in 
terms of its characteristics. 

The learning material is compiled and edited content from different resources. 
The sources have been recognized as acknowledgement and also in reference section. 


The content has been presented keeping Gagne’s nine events of instructions. 


The course would be helpful for the policymakers and researchers at central, 


state, district and institutional level. 


Veera Gupta 
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OVERVIEW OF EDUCATIONAL POLICY IN INDIA 
a. Objectives: 


After going through the unit, the reader shall be able to: 
1. Describe Role of government of India in formulation of educational policy. 
List salient features of recommendations of various Committees and Commissions. 


Link various changes in the educational system due to changes in the policy. 


ger Ste it 


Understand the basis of division of educational subjects in the union, state and concurrent 


list. 
b. Introduction 


In the present day world education determines the level of prosperity and welfare of 
the people. This is more so in a developing country like India. If the pace of national 
development is to be accelerated there is need for a well-defined, bold and imaginative 


educational policy. 


To understand the system of education prevalent in the country today, the origin can be 
traced to the beginning of the nineteenth century. Thereafter a number of Committees and 
Commissions were appointed from time to time by the Government in the pre-independence 
period to suggest appropriate changes in regard to the educational policy in view of the political 
and social changes taking place in the country. Education, however, acquired a new significance 
after Independence and both the planner and the administrator have felt the need for a national 


system of education suited to the needs and aspirations of free and modern society. 


The nature and content of the National Educational Policy as well as the manner and 
success of its implementation depends largely on the role of the Government of India has played 
in education. In this chapter, we shall briefly discuss the role of the Government of India in 


education as it has evolved over the years. 


c. 


Indian people in 1813. But Government of India did not then exist; the 
authority of the Governors of the three Presidencies of Bengal, Bombay and 
Madras was supreme and subject only to the control of the Court of Directors | equivalent to 


of the Company in England. Between 1813 and 1833, therefore, education saying that 


The Role of the Government of India in Education 


i) 


Pre independence period 


The East India Company was made to accept responsibility for the education of the 


In the modern | 
parlance, this is 


‘education’ ceased 


was an exclusive responsibility of the Provincial Governments. This situation " b 
O € 
however changed completely in 1833 when the Charter Act of that year 


authority to govern all the Company's possessions in India. This was basically became 
exclusively 


a political decision taken with the object of creating a powerful and supreme Central subject 


agency within the country itself to strengthen the appropriations there from. 


At one stroke, the Provincial Governments thus lost all their authority over ne 


education and became merely the agents of the Government of India to administer it on its 
behalf. They could not create a single post, however low, nor sanction a rupee of expenditure, 
nor make any change in policies, however small. In the modern parlance, this is equivalent to 
saying that ‘education’ ceased to be an exclusively State subject and became an exclusively 
Central subject. This system of a total centralization of authority continued till 1870. It would 
have been almost unworkable but for the fact that, In actual practice, the Government of India 
allowed a good deal of freedom to the Provincial Governments to adjust educational policies to 
their local needs and respected their proposals and advice. 

During this period, there were two main occasions when the Government of India issued 
orders, which, in substance, were tantamount to the enunciation of a National Policy on 
Education although this expression was not then used. The first refers to the orders issued by 
Lord William Bentinck, on the basis of a Minute recorded by Macau- lay, that the objective of 


modern education in India ought to be the promotion of European literature and science among 


exclusively State 
created the office of the Governor-General of India and vested in him the sole | subject and 


the people of India and that English should be the medium of education.’ The second was the 
Educational Despatch dated 19th July 1854 issued by the Court of Directors. It is a long 
document of 100 paragraphs, which laid down the broad principles and programmes for 
educational development in India. It confirmed the orders issued earlier by Lord Bentinck. This 


policy statement continued to govern all educational developments in India till 1882. 


In 1870, Lord Mayo initiated a system of decentralization and delegation of authority. 
The centralized administration introduced in 1833 had served its purpose, the whole of India had 
been conquered, the so-called Sepoy Mutiny quelled, and the British rule fully consolidated in all 
parts of the country. On the other hand, the weaknesses of the system, which led to delays and 
wastefulness in expenditure and created an irresponsible attitude in the Provincial Governments 
came to the surface and began to gall. A move for decentralization was, therefore, inevitable, and 
it came none too soon. 

Between 1870 and 1921, most of the administrative and financial authority over 
education was gradually transferred to the Provincial Governments, and the Government of India 
merely retained a theoretical right of the control and general supervision as a sequel to its 
ultimate responsibility to the British Parliament. The Government of India did not hesitate to 
intervene, to review educational progress, and to issue such directives to Provincial Governments 
as it felt to be necessary. The first such occasion to intervene arose when there was an insistent 
demand that the progress of education in India since the Educational Despatch of 1854 should be 
reviewed. The Government of India, therefore, appointed the first Commission on Education in 
India, viz. the Indian Education Commission (1882) and on the submission of its report, issued 
orders laying down a new National Policy on Education. These policies continued to be in force 
till the end of the nineteenth century. 

The second occasion for a strong and sustained intervention arose when Lord Curzon 
became the Governor General of India. He was of the view that Indian education had grown to 
fast at the secondary and university stages, that its administration had become flabby because of 
undue freedom given to Indian private enterprise, that standards had deteriorated and that the 
uncontrolled expansion of secondary and higher education was leading to indiscipline and 
dissatisfaction against Government. He was, therefore, of the view that the Government of India 


should no longer be a ‘king log' and that a policy of intensive central interest in education must 


be enunciated and sustained. He created the office of the Director-General of Public Instruction 
in India under the Central Government (1897). He also convened a Conference of the Directors 
of Public Instruction in the Provinces at Simla (1900), appointed the Indian Universities 
Commission (1902), passed the Indian Universities Act (1904) in the Central Legislature, and 
issued the Government Resolution on Educational Policy in 1904. He also initiated a system of 
large Central grants to the Provinces for educational development and these continued to be in 
vogue for several years afterwards. 

An Indian Education Service (IES) was also created in 1897 and its officers held all key 
posts in the Education Departments. A second Government of India Resolution on Educational 
Policy was also passed in 1913. The two Resolutions of 1904 and 1913 may also be described as 
National Policies on Education and form a continuing sequence with the orders of Lord Bentinck, 
the Educational Despatch of 1854, and the Resolution of the Government [of India on the 
Recommendations of the Indian Education Commission (1884). 

the system of Central grants, which had made such a useful contribution to educational 
development, also disappeared. 

This period of active Central contro! and financial support of education came to an end in 
1921 when, under the Government of India Act, 1919, education in the Provinces was transferred 
to the control of Indian Ministers responsible to legislatures with an elected majority. As a 
corollary to this basic decision, the role of the Government of India (which continued to be 


responsible to the British Parliament) in education had to be limited to the 


The system of 
Central grants, 
which had made 
such a_ useful 
contribution to 
educational 

development, 

also disappeared. 


minimum. Under the Government of India Act, 1919, therefore, the 
responsibilities of Government were divided into Central and Provincial lists 
in the first instance and then the latter were divided into reserved and 
transferred. Because of various conflicting proposals made on the subject, 
education was treated as partly all- India, partly reserved, partly transferred 
with limitations and partly transferred without limitations. 

Except for the matters reserved for the Government of India the whole 


of the education was transferred to the control of Indian Ministers with one 


exception, viz. education of Anglo-Indians and Europeans was reserved. There were also some 
restrictions on the control, which the Ministers could exercise in certain matters (e. g. the IES). It 


was also decided to stop all further recruitment to the IES so that the 'service would fade out 


within a few years. Needless to say the system of Central grants, which had made such a useful 


contribution to educational development, also disappeared. 


Policy underlying the Act of 1919 was continued with some modifications, which 
reduced the authority of the Government of India over education and correspondingly increased 
that of Indian Ministers over it at the provincial level. The actual provisions 


made in the Government of India Act, 1935, abolished the old distinction | No Government of 


between reserved and transferred subjects and the limitations placed on the India Resolution 
on National 


Educational Policy 


powers of the education ministers. The whole of education thus became a 
transferred subject, except the few matters, which were reserved for the | was issued for 55 


Government of India. years between 1913 
and 1968. 


With these radical changes in the constitutional position, the question 


of the Government of India issuing any Resolutions on the National Policy on Education did not 
even arise; and in fact, no Government of India Resolution on National Educational Policy was 
issued for 55 years between 1913 and 1968. Ilowever, attempts were made to strengthen the 
advisory and coordinating role of the Government of India. The quinquennial reviews of 


education in India continued to be published. 


A Central Advisory Board of Education consisting of all Education Ministers in the 
Provinces was first constituted for this purpose in 1921. It was however abolished in 1924 as a 
measure of economy. It was revived again in 1935; and its one great achievement was to prepare 
the Post- War Plan of Educational Development (1944). It proposed to create, in a period of 40 


years, an educational system in India, which would be like that in England in 1939. 
ii Post independence period 


The subject came again for discussion in 1950 when the Constitution was being adopted 
on the basis of Federal Government at the Centre. It could have been possible at this time to 
define a more effective Central role in education. But the general public opinion was strongly in 
favour of vesting almost all authority over education in the States. Once again, therefore, the 


basic policy of the Government of India Act, 1919, was continued with one change, viz. certain 


additional powers were now vested in the Centre. In fact, the Constitution makes Education 
basically a State subject (Entry 11 of the State List), subject to certain powers reserved in the 
Central (Entries 63-66) and Concurrent Lists (Entry 25). You will read in detail about 


constitutional provisions in unit two. 


If history is to be written in modern parlance, it is interesting to note that education has been in 
the exclusively Central list between 1833 and 1870, in the Concurrent list between 1870 and 
1921, in the exclusive State list between 1921 and 1947, mostly in the State list with some 
special powers to the Centre between 1947 and 1976, and in the Concurrent list from 1976 


onwards. 


National Educational Policy Resolution (1968) 


‘There was a formal and official statement on the National Policy on Education, which 
was expected to guide educational planning, and implementation. ‘lhe educational planners of 
1966-78 could look up to the report of the Education Commission (1964-66) which the 
Government of India had broadly accepted and also to the National Policy on Education (1968) 


which had been issued after due consideration of its proposals. It is not 


necessary to discuss here the recommendations of the Education Commission in | At this time, 
education was 
not in the 
concurrent list 
and there was 


little 


detail. It is however essential to refer, at the very outset, to one crucial 
recommendation. The Commission was of the view that one of the main reasons 
for the unsatisfactory progress of education between 1947 and 1965 was that the 


country did not have a comprehensive and rational policy of education so that 


basic educational issues were being decided in an ad hoc manner anti on possibility of 
con passing a 
considerations of expediency and class interests. It therefore recommended that | yJational 


a national system of education should be created in the country in a period of 20 | Education Act 
years (1966-86) and that, for this purpose, "the Government of India should 
issue a statement on the national policy in education which should provide guidance to the State 
Governments and the local authorities in preparing and implementing education plans in their 
areas. The possibility of passing a National Education Act may also be examined" (para 18-58). 
At this time, education was not in the concurrent list and there was little possibility of passing a 


National Education Act in Parliament. However, the Government of India accepted the remaining 


part of this recommendation and decided to issue a statement on the National Policy on 


Education. 


The decision of the Government of India to issue a Statement on National Policy on 
Education was therefore widely welcomed. A statement on the national policy on education was 
last issued as far back as 1913 and that this Statement was now being issued after a lapse of 55 


years. 


It took nearly two years to go through all these steps for the formulation of the 1968 
policy. The report of the Commission was received in June 1966. By the time it was printed and 
circulated and comments and suggestions on it began to come in elections to the Parliament 
became due and it was felt that the third step of appointing a Committee of Members of 
Parliament could best be taken after the elections. The Committee of Members of Parliament 
therefore could only he appointed in 1967. Thereafter, it took considerable time for discussions 
to be held in both Houses of Parliament and for the whole matler to be considered and decided by 
the Cabinet. The Government Resolution on National Policy on Education could therefore be 


issued only in 1968. 
National Education Policy 1986 and POA 


The Constitutional Amendment of 1976, which included Education in the Concurrent List, was 
a far-reaching step whose implications-substantive, financial and administrative, required a new 
sharing of responsibility between the Union Government and the States in respect of this vital 
area of national life. While the role and responsibility of the States in regard to education 
remained essentially unchanged, the Union Government accepted a larger responsibility to 
reinforce the national and integrative character of education, to maintain quality and standards 
to study and monitor the educational requirements of the country as a whole in regard to 
manpower for development, to cater to the needs of research and advanced study, to look after 
the international aspects of education, culture and Human Resource Development and, in 
general, to promote excellence at all levels of the educational pyramid throughout the country. 


Concurrency signified a partnership, which was at once meaningful, and challenging. 


The prime Minister, Shri Rajiv Gandhi, in his address to the nation on January 5, 1985 
touched upon the need for new National Education Policy. The Prime Minister set up a three- 
member informal group headed by Prof. P.N.Srivastava, Vice-Chancellor of Jawaharlal Nehru 
University to advise him on educational plans and policies. The other members of the group 
which suggested reforms in education and laid down priorities were Dr.P.L. Malhotra, Director 
of the National Council of Educational Research and Training and Dr. Moonis Raza, former 


Director of the National Institute of Educational Planning and Administration. 


The National Policy on Education envisaged a meaningful partnership between the 
Union Government and the States in implementing the Policy. The response of the State 
Governments and U.T. Administrations had indeed been very encouraging to all major 
programmes like Operation Blackboard, restructuring of Teacher Education, Vocationalisation 


etc. which were in the process of implementation during that current year. 


Programme of Action 1986: 


The Parliament during the Budget Session in 1986 discussed and adopted the "National Policy on 
Education 1986". In the first place, 23 Task Forces were constituted and each was assigned a 
specific subject covered by the National Policy on Education (N.P.E.). Eminent educationists, 
experts and senior representatives of Central and State Governments were associated with these 


Task Forces, 


The Task Forces were requested to examine the present situation in respect of the 
subjects assigned to them and to elaborate the implications of the specific statements contained 
in the N.P.E. The Task Forces were also expected to project the actions that would be necessary 
and indicate the broad targets and the phasing of the programmes. They were requested to 


indicate the broad financial implications with reference to each phase. 


The Task Forces accomplished their work with great care. They submitted their reports in July 
1986. These reports were discussed in a series of meetings taken by the Minister of Human 
Resource Development. After these discussions were completed, a conference of Education 


Secretaries of the State Governments and U.T. administrators was convened on the 20th July, 


1986. Suggestions received during this meeting were carefully considered and the Programme of 


Action was prepared with reference to the main areas covered in N.P.E. 
iv) Modified policy in 1992 and POA: 


On the 7th May, 1990 the Government of India announced the appointment of a Committee 
‘to review the National Policy on Education (NPE), 1986. The resolution of the Government 


reads as follows on the need of revision of the policy: 


"Despite efforts at social and economic development since attainment of independence, a 
majority of our people continues to remain deprived of education. It is also a matter of grave 
concern that our people comprise 50 per cent of the world's illiterate, and large sections of 
children have to go without acceptable level of primary education. Government accords the 
highest priority to education both as a human right and as the mcans for bringing about a 
transformation towards a more humane and enlightened society. There is need to make education 
an effective instrument for securing a status of equality for women, and persons belonging to the 
backward classes and minorities. Moreover, it is essential to give a work and employment 
orientation (o education and to exclude from it the elitist aberrations, which have become the 
glaring characteristic of the educational scene. Educational institutions are increasingly being 
influenced by casleism, communaliym and obscurantisun and it is necessary to lay special 
emphasis on struggle against this phenomenon and to move towards a genuinely egalitarian and 
secular social order. The National Policy on Education (NPE), 1986 needs to be reviewed to 
evolve a framework which would enable the country to move towards this perspective of 


education." 


These concerns were not new except the Right to Work now being sought to be enshrined 
in the Constitution. They were there when The Challenge of Education was written in 1985 and 
The National Policy on Education formulated in 1986. Programme of Action (POA), 1986 was 
updated in 1992. The modified policy envisaged a National System of Education to bring about 
uniformity in education, making adult education programmes a mass movement, providing 
universal access, retention and quality in elementary education, special emphasis on education of 


girls, establishment of pace-setting schools like Navodaya Vidyalayas in each district, 


10 


vocationalisation of secondary education, synthesis of knowledge and inter-disciplinary research 
in higher education, starting more Open Universities in the States, strengthening of the All India 
Council of Technical Education, encouraging sports, physical education, Yoga and adoption of 
an effective evaluation method, etc. Besides, a decentralised management structure had also been 


suggested to ensure popular participation in education. 


Programme of Action 1992: 


The POA laid down a detailed strategy for the implementation of the various policy 


parameters by the implementing agencies. 
e. Five year plans: 


Five-year plans are also policy statements with respect to allocation of funds to various 
schemes and programmes. Up to Fifth Year Plans, education was taken to be a social service 
rather than an input in the development process. There was a change in the emphasis beginning 
with the Sixth Plan wherein it has been considered to be pivotal in the social and economic 
development of the country through development of human resources. POA 1992 was more 
practical and action oriented. Five-year plans are aligned as per policy statements. After 1992 no 
policy statement has been issued. Subsequently policy implications are being reflected in the 


eighth, ninth and tenth five year plans. 


As we have read that education is in the concurrent list, both the Centre and the States are 
empowered to enact through their respective legislature and to make codes, rules, regulations and 
policies that provide educational services to the people. In the next unit we shall study the 
definitions and components of the Policy. We shall also study policy formulation procedures 


adopted and required to be adopted to formulate a good policy. 
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f. 


1. 


iS) 


Exercises: 


Mark True or False: 


i English as medium of instruction was introduced in India by the orders issued by lord 


Bentick 
mY &® 


ANS: (T) 
ii The Education Despatch of 1854 is in substance is equivalent to second National Policy 


on Education. 


(WU) ©) 


---ANS+(B)~=- 


iii Entry 11 of the constitution makes the education state subject. 


is es 
Ans: (T) 


Prepare a time line indicating transfer of education from central Government to State 
government and vice versa also Indicate dates and name of the policy /order. 
Name any three educational policies with their salient features from 1870- 1920 period. —' 


Point out schemes, which are not part of the 1992 policy but are operative due to budget 


allocation in tenth five-year plan. 
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CHAPTER II 
POLICY FORMULATION IN INDIA 


a. Objectives: 
After going through the unit, the reader shall be able to 
1. Describe components of an educational Policy 
2. Differentiate policy-making bodies in the center and state. 
3. Examine different approaches used in the policy making. 
4. List characteristics of educational policy 


b. What is a Policy? 


Policy can be defined as statements that have purpose of guiding decision- 


making and actions. When people refer to Policy they are 
usually referring to the set of normative statements, 
systems, organizational structures and decision-making 
processes. Policy Statements provide system-wide 


guidance, procedures, and requirements. These policies are 


based on external and internal mandates, laws, and 
regulations. Many people see the creation of public policy 
aS synonymous with its implementation. [lowever, while 
creation of public policy is important, the success of policy 


lies in how well it is implemented. 


“Devising the policy 
was our job, but 
now it’s up to the 
directorate of 
education to ensure 
that 
recommendations 


are implemented.” 
Ashok Ganguli, Chairman 
CBSE, on Nursery 
admissions: Parents told to 
lodge Complaints, 13.12.06 
TOI 


Policy analysis felt that study of policy impact is closely related to policy 


formulation. If formulation is faulty results cannot be expected. As a result 


policy formulation started emerging as body of knowledge. In the third unit 


we shall discuss evaluation of policy from both the perspective i.e. 


formulation and implementation. In this unit we shall discuss components and 


formulation procedures of a policy. 


b. Components of a policy 


A Policy is a plan or course of action, of a government, political party, 


intended to influence and determine decisions, actions, and other matters. ' The 


process of policymaking includes the manner in which problems get 
conceptualized and brought to government for solution; governmental institutions 
formulate alternatives and select policy solutions; and those solutions get 
implemented, evaluated, and revised. In democratic governance, Constitution is 
the foundation of the policy in any area and in any country. Policy rests on public 
opinion, proper funds allocation, administrative machinery and political will. In 
order to get results all the four should work in coherence. How ever analysis of 


policy impact studies show absence of one or more components. 


Components of a Policy 


(Diagram 2.1) 


Public opinion means contribution of the public/ stakeholders in the 
making of the policy and also control of the public on the issues of the policy. In 
education, the policy-making bodies seek public opinion through media, letters 
and voting. At times public opinion is not sought directly but through 
representatives of public by way of inviting experts on committees. Involvement 
of Public is also desirable by way of control. Control is left to public with respect 
to management of education. It could be through village committees and 
Panchayats in the localities. 

Consultation has recently been recognized as an essential element of public 
policy making. Organization for Economic Cooperation and Development 


(OECD) is working to construct this knowledge base. For the policy formulation 


a 


legal, and institutional frameworks are still under development. If you study the 
policy formulation procedure adopted by various agencies in India, you will find it 
at variance. Some time it is on the basis of recommendation of a committee or 
commission and some times it is by issuance of a order. Therefore you will find 
overlap in the concepts of policy, regulations, rules and orders while studying 
literature on policy. 

Administrative Machinery is key to successful implementation of the 
policy. Clear-cut identification of the agency and their functions are also part of 
the policy making. This exercise got started in India with the preparation of 
Programme of Actions with each policy. It is also necessary to identify and 
delegate responsibility to the centre, state, district, and block level functionaries. 
It allows the policy to retain national character but have local shape. You will read 
that policy-making body may not be the implementing agency. Impact studies on 
policies bring out missing link among these ayencies. 

Funds allocation and distribution to administrative agencies is third aspect 
of policy making. Good policies remain on paper without proper allocation or 
distribution system of funds. Source of the funds also influence policy-making 
procedures. If funds are to be given out of central Government the issue becomes 
of the central nature and if funds are lo be drawn oul of slale povernment the issue 
is of the state government. Therefore the education was being transferred between 
centre and states and later as concurrent list subject. Fund allocation and timely 
release of funds is one of the main reason of success and failure of many Schemes. 
Some times funds allocation orders are also treated as policy statements. Such as 


five year plans. 


The political aspects of the problem also need some attention. It is obvious 
that the chances of enunciating and implementing a strong national policy are 
greater when the same political party is in power at the Centre and in the States (or 
when the different political parties in power agree on a programme of national 
significance). This, for instance is the situation in the USSR where, inspite of a 


very large constitutional delegation of authority over education to the States, a 


uniform national system of education has been created and maintained by the 
Communist Party. In India also, national policies in education have depended for 
their success, not so much on Constitutional provisions but on the strength of the 


ruling party and the capacity of the national leadership to assert itself. 


c Policy formulation process: 

Literature on the 
policy making 
growing branch of knowledge. More and more concepts of | process is far less 


As we have read that policy formulation is a recent and 


policy formulation are being integrated into policy-making than the literature 
; F AAs . on policy issues 
exercises and research design. It is interesting to note that 

the literature on the policy making process is far less than 
the literature on policy issues and on its analysis. In this section we shall study the 
procedure to be followed for policy making. Any policy goes through the 


following cycle: 


8. Evaivation akinveivn t 


7. Implementation 3. issue 


6. Cholce 
6, Alternatives 


8. Consequences 


(Diagram 2.2) 
(Adapted from Charles Gratto, 1973 
Issue Evolution-Educational Intervention Model 


Source: Singletary, Loretta. Cooperative Educator, University of Nevada Reno Cooperative Extension Service. WESTERN 
RESOURCE ISSUES EDUCATION SERIES - No. 2, Fact Sheet 97-24, "Selecting a Process for Issues Education." ) 


Stage 1 - Awareness: 
Citizens and experts with an interest or stake in an issue (stakeholders) 


increase their awareness of the issue. Awareness emerges through informal 


and on its analysis 


ee 


discussions, sporadic complaints, or in extreme circumstances, litigation forcing 
action on an issue. In the awareness stage, the process offers the public an initial 
opportunity to exchange viewpoints about a concern(s). This exchange helps 
citizens clarify concerns by legitimizing their complaints, hearing about how 
others are affected by the same issue, and separating rumor from fact. 
Stage 2 - Involvement: 

Other stakeholders are identified who are affected by the issue but are not yet 
involved in discussions. Citizens may also identify information specialists to 
provide facts about the issue and who might help identify other stakeholders. 


“Citizens are increasingly demanding greater transparency and accountability 
from their Governments, and want greater public participation in shaping 
policies that affect their lives. Educated, well-informed citizens expect 
governments to take their views and knowledge into account when making 
decisions on their behalf. Engaging citizens in policy-making allows governments 
to respond to these expectations and, at the same time, design better policies and 
improve their implementation. ’" 


Stage 3 - Issue Clarification: 

Clarifying the concern and framing it formally as a public issue is the goal in 
the third stage of the issue evolution cycle. Stakeholders may exchange 
individual perceptions of the problem through focus group interviews, panel 
discussions, public forums (whole group input), and/or study groups. 
Knowledge-based experts on the issue may he invited to conduct or coordinate 
scientific research and share research results with the public. 

Stage 4 - Alternative Identification: 

As the issue is clarified through the educational process, stakeholders identify 
and/or create alternatives to resolve the issue. In addition to scientific or technical 
information provided by subject matter specialists, stakeholders may conduct their 
own research to identify alternatives. Citizen research may include: reviews of 
journal articles, books, videos, citizen surveys, and case studies of areas with 
similar issues. Ideally, the alternatives generated are based on factual, objective 
information combined with an effective exchange of individual views, ideas, and 
values. 


Stage 5 - Consequence Analysis: 


Experts examine carefully the consequences of the alternatives created in stage 
4. This involves looking at the measurable costs and benefits of alter-natives in 
terms of, for example, time, money, technical feasibility, and human and physical 
resources required. In addition to economic consequences, social consequences 
must be considered as well. Potential losses to public welfare are difficult to 
measure, but provide important information to consider when weighing 
consequences of public action. 

Stage 6 - Choice: 

After careful consideration of alternatives and consequences of a particular 
action, stakeholders can provide informed input as how to address the issue. In 
making a choice, stakeholders learn or improve their understanding of how 
public choice is shaped into public policy. This may involve learning how to 
influence elected officials as well as individuals who influence decisions behind- 
the-scenes. Ideally, stakeholders are in agreement that the choice represents the 
best possible way of addressing the issue. They must be open, however, to 
working through conflicts that might arise among disagreeing interests. Hard-line 
advocates of a particular choice must learn that there are advantages in 
negotiating and collaborating with their opponents. If they refuse to negotiate, the 
issue may end up unresolved. Therefore, striving for a solution that satisfies all 
interests is of interest to all stakeholders. 

Stage 7 - Implementation: 

In this stage, the choice is implemented in the form of a policy or formal 
agreement of understanding. Stakeholders need to understand how the agreement 
or new policy will be implemented. They need to look for changes in public 
opinion that might occur during its implementation. Individual concerns may arise 
during implementation that includes, for example, possible third party injuries. 
This possibility emphasizes the importance of including a broad and diverse array 
of stakeholders in the awareness and involvement stages of the issue evolution 
cycle. It also underlines the importance of examining carefully the consequences 
of given alternatives. 


Stage 8 - Evaluation: 


This final stage of the cycle evaluates the effectiveness of the choice or 


implemented policy. At this stage stakeholders may ask: 


Is the policy or action is taking care of the problem? 
Does the public agree that the policy is effective? Why? 
Ts it perceived generally as ineffective? Why? 

What can be done to improve it? 


The final stage offers an additional opportunity to evaluate the entire We 


issue and education process. Stakeholders may ask: awareness is 
growing _ that 
e What happened at each stage? policy 
e Why did this happen? formulation is 
e What else might have happened? also a 
e Has the situation improved? : 
e What can we do to improve the situation? methodical 3 
process, its 
In a sense, stage 8, offers a chance to begin the cycle anew--with documentation 


is also 
increasing. 


more information and experience begin clarifying concerns. 


Information with regard to procedure followed for making a policy is 
also not available for the earlier policies. As the awareness is growing that policy 
formulation is also a methodical process, its documentation is also increasing. It 
is evident from the information available on the procedure followed for the three 


national policies of the independent India. 


d. The procedure followed for 1968 Policy: 


The Government of India appointed education commission known as Kothari 
commission to prepare detailed report. The Report was circulated to all 
Ministries of the Government of India, to the State Governments and Union 
Territories and to the Universities for their comments and suggestions. The 
Report was also considered in details by a Committee of Members of Parliament 
representing both Houses and all political parties. The Report of the Education 
Commission along with the report of the Committee of Members of Parliament, 


was discussed in both the Houses of Parliament; and on the basis of these 


discussions, a Government Resolution on National Policy on Education was 


issued. 
e Procedure followed for the NPE 1986: 


The Government of India announced in January, 1985, that a new Education 
Policy would be formulated for the country. A full appraisal of the existing 
educational scene was undertaken and a document, entitled, "Challenge of 
Education" was brought out in August 1985. There has been a countrywide debate 
on the document, marked by keen interest, and enthusiasm. The views and 
suggestions received from different quarters were carefully studied and new 


policy was formulated. 
f. Procedure followed to prepare National Education Policy 1992: 


The National Policy on Education was adopted by Parliament in May 
1986. Ramamurty Committee was set up to review the policy in 1990. 
Janardhana committee was set up to suggest modifications in the NPE based on 
the recommendations of Ramamurty committee. The report of this committee was 
considered by Central Advisory board of Education in 1992 and accepted the NPE 
of 1986 with a few modifications. The policy has stood the test of times and is in 
place even today. It may be attributed to large scale consultations on the policy. 
People sent their opinions to the Ministry of Education/Ministry of human 
Resource Development. A large number of, them had chosen the ‘press' to express 
their concerns/opinions regarding New Education Policy. 

The Press in fact was a significant medium for providing feedback to the 
Government. This feedback came in many forms. Firstly it got expressed in the 
form of the organised view of the concerned agency through editorials. Secondly, 
letters to editors in the different papers air the people's reaction on various issues 
and lastly, the various news items contained in the media provide an opportunity 
to the masses to know the development regarding on-going debate in the entire 


country. 


It was thus obvious that it was the editorials, which were the most 
significant because they provided a critical evaluation of various issues contained 
in the pronouncement of various, dignitaries as well as in other documents. The 
role of the press became more important because it was not only the agency for 
airing views but was also a disseminating, channel and a forum for reaction and 
rethinking; on these reactions in the ultimate analysis it became a source of inflow 
for policy makers and planners. 

This analysis was based on clippings of the news papers/journals sent to 
Ministry of human Resources Development in order to provide some kind of 
feedback. The clippings received were classified into the following four categories 
i.e, editorials, articles, news items and letters to the editors. 


The frequency of clippings received against each category was as follows: 


i) Editorials 31 
ii) Articles 97 
iii) News, Items 130 
iv) Letter to Editors 17 
Total No. of Clippings 275 


‘lhe entire specturum of clippings belonged to only three languages i.e. Hindi, 


English and Marathi. Their distribution was as follows: 


Hindi English Marathi ‘Total 


Lditorial 14 10 7 31 
Articles 35 41 21 97 
News Items 70 43 17 130 
Letters to Editors - 13 4 17 
Total 119 107 49 275 


These clippings belonged to newspapers published from different regions 
of the country. However, they had not been derived on the basis of any sampling 
technique. As mentioned earlier, only those clippings which were sent to the 
Ministry of Human Resources Development and received by NIEPA had been 
taken note of Therefore, a majority of clippings belong to the papers published 


from Delhi, Bihar, Maharashtra and Madhya Pradesh. The State wise frequency is 


given in the following table: 


1) Delhi 66 
2) Bihar 51 
3) Maharashtra 51 
4) Madhya Pradesh 36 
5) Uttar Pradesh 17 
6) Rajasthan 12 
7) Karnataka 8 
8) Tamil Nadu 8 
9) Andhra Pradesh 7 
10) West Bengal 5 
11) Chandigarh 5 
12) Gujarat 4 
13) Assam 1 
14) Punjab 1 
15) Andaman & Nicobar Island 1 
16) Goa, Daman & Diu 1 
17):U-P. i 
Total: 275 


It is evident that the first four States/Union Territories account for about 
three-fourths of the press clippings. One may have an argument over such a 
squeezed distribution taken for this analysis but it should not act as an 
impediment in discerning the significant trend. 

The role of the press in the formulation of National Policy on Education 
should be viewed in terms of direction it could provide to the Government in 
various stages of policy evolution. Therefore the major focus of the press had been 
"on issues, which had a bearing on policy and planning. The entire spectrum of the 
issues discussed in the press-media had been categoriesed into the following 
themes: 

1. Goals of Education 

2. Structure of Education Content and Curriculum 
3. Content and Curriculum 

4. Language 

5. Evaluations and Examination 

6. Planning and Management 


7. Financing of Education 


8. Teachers 

9. Social justice 

10. Delinking Degrees from jobs 

11. Access to higher Education 

12. Vocationalisation of Education 

13. Discipline and Politics 

14. Infra-structure Facilities 

15. Adult and Non-formal Education 

Issues pertaining to various themes have not found a uniform place in the entire 
analysis. In fact they should not be. The representation of themes in the entire 


clippings is as follows: 


Themes No. of clippings % out of total 
containing response No. of Clipp- 
on individual themes ings (275) 

Planning & Management 133 48.36 

Teachers 672 4.36 

Content and Curriculum 59 21.36 

Vocationalistion a 19.27 

Goals of Education 44 16.00 

Discipline and Politics 40 14.54 

Financing of Education 36 13,09 

Evaluation and Examination 35 12.72 

Infrastructure facilities 26 9.45 

languages in Education 22 8.00 

Social Justice 18 6.45 

Delinking Degree from job 14 5.09 

Access to higher Education 12 4.36 

Non-formal and Adult Education 10 3.63 

Structure of Education 9 3.27 


Besides these three comprehensive policy statements, resolutions, schemes, and 
guidelines are issued from time to time on individual concerns. Procedure for 
formulation of policy should remains the same whether it is comprehensive 


statement or one individual issue. 


g. Educational policy making bodies and their relationships 


In the first chapter we have studied the history of Indian education policy, which 
has given us fair idea of the policy-making bodies. In India, Centre, State and 
Local bodies have formulated policies solely or in collaboration. The relationship 


and distribution of subjects is as laid down in the constitution. 


1. Constitutional provisions 


Constitutional provisions are the foundation of the policy. Under the 
Constitution as originally enacted, education was primarily a State subject. The 
relevant entry (Entry 11) in the State List read "Education including universities 
subject to the provisions of entire 63,64,65 and 66 of List I and 25 of List III" 
Thus education was exclusively a responsibility of the States, the Central 
Government being concerned directly with certain areas like coordination, 
determination of standards in technical and higher education etc. In 1976, by a 
constitutional amendment education became the joint responsibility of the Central 
and State Government. ‘I'he Constitution (Forty-second Amendment) Act, omitted 


Entry 11 from the State List and amplified Entry 25 in the Concurrent List. 


The implication of this amendment was that education had become a 
Concurrent Subject which enables the Central Government to legislate in such 
fields as, for example, school education. Apart form this concurrent sphere; the 
Constitution places certain educational matters within the exclusive jurisdiction 
of the Union. These are: maintenance of Central Universities and institutions of 
national importance, declared as such by Parliament by law; all Union agencies 
and institutions for professional, vocational or technical training or for promotion 
of special studies or research; and the coordination and determination of 
standards in institutions of higher education or research and scientific and 
technical institutions (Entries 63,64,65 and 66 of List I) 

Article 28 provides that no religious instruction shall be provided in any 
educational institution wholly maintained out of State funds. Article 29 further 
provides that no citizen shall be denied admission into any educational institution 


maintained by the State or receiving aid out of State funds, on grounds only of 


religion, race, caste, language or any of them. According to article 30, all 
minorities, whether based on religion or language, shall have the right to 
establish and administer educational institutions of their choice. In addition, the 
State shall not, in granting aid to educational institutions discriminate against any 
educational institution on the ground that it is under the management of a 
minority, whether based on religion or language. 

Besides, some of the Directive Principles, set out in Part IV of the 
Constitution, also have a bearing on national educational policy. For instance, 
article 45 enjoins that the 'State' shall endeavour to provide for free and 
compulsory education for all children until they complete the age of fourteen 
years. 

Article 46 requires the ‘State’ to promote with special care the educational 
interests of the weaker sections of the people, which include, in particular, the 
Scheduled Castes and Scheduled Tribes. 

Article 350A places a special responsibility on the Government to 
safeguard the interests of the children belonging to linguistic minority groups and 
to see that they have adequate facilities to receive at least primary education 
through their own mother tongue. Article 351 further places a special 
responsibility on the Government of India for promoting the spread of Hindi 
language and for its development so that it may serve as a medium of expression 
for all the elements of the composite culture of India. 

Yet another provision in the Constitution, which has an indirect but 
significant bearing upon the role of the Government of India in education, is entry 
20 of List II], which is concerned with "Economic and Social Planning". 
Education Planning being an essential element of economic and social planning, 
the Government of India and the State Governments have to work together in 
preparing and implementing the national plans for the reconstruction of education. 
The Government of India was also responsible under the Constitution for the 
administration of the Union Territories and has for this purpose executive and 


legislative authority for all subjects including education. 


Next to the constitution, State Policy is articulated through legislations. A 


few states have also formulated State Education Policies. Besides that, there are 


twenty-five regulatory bodies established by the acts of parliament to formulate 


policies on the subjects allocated to them. These are: 


1. 


The University Grant Commission Act 1956 


. The All India Council For Technical Education Act, 1987 


The National Council for Minority Educational Institutions 
Act, 2004 

The Copy right Act, 1957 

The apprentices Act, 1961 

Bar Council of India, 1961 

Central Council of Homeopathy, 1973 

Central council of Indian Medicine, 1970 


Council of Architecture, 1972 


. Dental council of India, 1948 

. Distance Education Council, 1985 

. Indian Nursing Council, 1947 

. Medical Council of India, 1956 

. National Council for Teacher education, 1993 
. Pharmacy Council of India, 1948 

. Rehabilitation Council of India, 1992 

Te 


Indian Council for Agricultural Research, 


. State Councils for Higher Education, 

. Veterinary Council of India, 1984 

. Directorate General of Shipping, 1958 

. The Institute of Chartered Accountants of India, 1949 
D2: 
23. 


The Institute of Company Secretaries of India, 1980 
The Institute of Cost and Works Accounts of India, 1959 


Educational Policy is formulated at five different levels in any | formulated 
state: secretariat, directorate, regional, district and institution. 
But at present policymaking is done generally at the secretariat | Jevels in any 
and directorate levels and some times at institutional and 
panchayat level. NUEPA has conducted surveys of education | directorate, 
administration in eighteen states of India. Reports of these 
surveys throw light on policies and policy-making bodies in 


each state. 


h. 


24. Department of Electronics, Govt. of India 
25. National Council for Rural Institute, 1995 


Educational 
Policy is 


at five 
different 


state: 
secretariat, 


regional, 
district and 
institution 


Characteristics of a national educational policy: 


The making of public policy for a country as large, populous and diverse as India 


is a more complex task than for country smaller in size. India is regarded as under 


performer with regard to economic targets and human development targets. It 


could be due to wrong policies or due to poor implementation of right policies. It 


could also be due to wrong policy making processes. We have studied the right 


policy making processes. If these processes are adhered to what attributes could 


be seen in a policy. ‘hese could be as follows: 


ie 
2: 
3; 


6. 


Policy is based on expert analysis of the problems and issues. 
Overlap and tradeoffs with other sectors are reflected in the policy. 
Opposing points of view within and between sectors, are articulated, 
analyzed and considered. Accordingly reactions anticipated and 

options identified. 

Policy decisions have legal authority. 

Implementers are involved in the policymaking but are not the 
controller of the policy. 


Policy offers choices within a time frame. 


In the third unit we shall study the methods of policy analysis from these quality 


perspectives. 


« 


i. Exercises 
ve Tick Mark True or False 


a) Changes in the Constitution may bring Changes in the Policy of the 
Country (1) (Ff) 


Ans: (T) 
b) Policy is not implemented if it does not have public concurrence. 
(I) (F) 
Ans: (F) 
c) Policy may or may not contain Implementation details. 
(I) (F) 
Ans: (T) 
d) Financial plans are policy statements. 
@ 
Ans: (T) 
2. Choose uny one educational policy and List its components. 
3. Read a few policies and derive the definition of a Policy in your own words. 
4. Read the following news item and write in 500 words what procedure 


the minister should follow to translate his plan into policy: 

“ Colas with or without pesticides are harmful for health and should not be consumed. We want all 
forms of junk food like pizzas, chips samosas and burgers banned from canteen” Ramadoss, 
Minister of health, TOI 15th Dec.06 


ery bok Ohta Ge oe A 
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Chapter -III 
RESEARCH METHODS FOR POLICY EVALUATION 


a. Objectives: 
After going through this unit reader shall be able to : 
1. Describe aspects of a policy we need to know about, and how should 
information about those aspects be collected. 
2. Differentiate between process and impact evaluation of policy. 
3. Examine the extent to which the research design is influenced by the 
parameters of the programme/policy. 
4. identify conditions for the process and impact evaluation to be used. 
b. Introduction 

Evaluations are used in a large number of fields and to answer a very wide range 
of questions about when and how interventions or treatments ‘work’. Although 
methodologies are to a large extent portable across subject areas, with, for instance, 
evaluations of health interventions using designs very similar to those used in the 
evaluation of social policies. 

Deciding on a design for an evaluation is a far from easy task. There are 
numerous approaches that can he used, and the decisions about which evaluation model 
or models to adopt will depend crucially on the questions of interest and the nalure of the 
policy or programme to be evaluated. At the very basic level, in designing an evaluation 


the key questions that will need to be considered are: 


_Do we need to know more about how the policy or programme operates on the 


ground? If yes, then what aspects do we need to know more about, and how should 
information about those aspects be collected? Process evaluation, which addresses these 


questions, is covered in Section 3 of this module. 


_Do we need to know what impact the policy or programme has in terms of 


desired outcomes? And, to answer that question, do we need to know what would 
happen, should it not be in place? That is, is it necessary to estimate the counterfactual? 


If yes, then how should the counterfactual be measured? The designs described in Section 


4 of this module cover the main methods that might be used. Sometimes, evaluations 
may only need focus on either the process or the impact of the policy or programme. 
More commonly, the research will involve both elements and thus both types of 
evaluation. Most of the examples used in this module are derived from evaluations 
carried out by the DWP, formerly the DSS, on active labour market programmes. 

The unit is divided into three main sections. Section 2 describes the main 
questions which can be answered by process and impact evaluations, and examines the 
extent to which the research design is influenced by the parameters of the programme and 
its timetable. Sections 3 and 4 take us into the detail of the two types of evaluation. 
Section 3 looks at process evaluation, with a particular focus on why it might be carried 
out and the possible research methods used. 

Section 4 is devoted to impact evaluation and, in particular, on methods of 
estimating the counterfactual. We focus on five methods, namely randomized trials, 
matched area comparisons, before-after studies, matched comparison group designs and 
difference-in-differences approaches. Cost-benefit analysis is also discussed. 

A key distinction is made in the module between process and impact evaluation. 
Another distinction commonly used in the literature, but which we only mention in 
passing in this module, is that between formative and summative evaluation. Formative 
evaluation is evaluation undertaken to provide information that will be used to improve a 
programme; summative evaluation is evaluation used to form a summary judgement 
about how a programme operated. The design issues are largely the same whether the 
evaluation is considered as formative or summative. 

c Deciding which type of evaluation is required 
i When should we use process evaluation? 

Listed below are various questions that we may want answered by a programme 
evaluation. If so, we will want to use some form or forms of process evaluation. Of 
course, the exact questions that need to be answered will depend on the policy to be 
evaluated and the objectives that policy makers have for the evaluation. As ever, it is 
important that the policy objectives are set out clearly so that appropriate research 


questions can be formulated. 


Finding out about service use 


q 


What is the level of awareness among the eligible population and the potentially 
eligible? How did people hear of the programme? Do they understand the key 


elements of the programme? 


Do all those eligible receive the programme? Who participates and why; who 


does not participate and why? 


_Do some that are not eligible receive the programme and if so, does this suggest 


that the target population is poorly defined or that programme delivery is poorly 


controlled? 


Finding out about service delivery 


_Is service delivery consistent with the programme's intention? 


Are all elements of the programme being delivered adequately and consistently? 


. How much change has occurred since implementation? 


How does provision compare with that before implementation of the programme? 


Does the programme comply with professional and legal standards? For example, 


are there appropriate complaints procedures? 


Identifying any variations in service delivery 


O 


_ Are services delivered according to different models or by different organisations 


It so, how do these compare? 


_Are programme resources or programme delivery consistent (and appropriate) 


across all geographical locations? 

_Do variations exist across locations or models which provide examples of best 
practice or which identify important interventions that should be considered 
elsewhere? Are there variations over time or for different groups? 

Are services focussed on those who are easier to reach at the expense of those 
who are harder to help? If so, what is the impact on the nature of the services 
provided and the net outcomes from these services? What would be the effect of 


shifting the balance between the easier and harder to help? 


Finding out about the organisation of the programme 


_Is the programme well organised? 


_ What are the main models of programme organisation and delivery? 


_ How well do different groups involved in delivery work together (in terms of 


different staff within delivery teams, and in terms of different programmes and 
agencies with which it must interact)? 


Looking at the programme’s resources 


_ Are adequate resources being used to deliver the programme? 


_Is programme staffing and funding sufficient to ensure appropriate standards? 


fm) _ Are programme resources used effectively and efficiently? 


_ Are costs per unit reasonable? 


! 
_ Are costs per outcome reasonable and are they offset by the benefits? 


t& 


ooking at participants’ experiences of the programme 


Are participants receiving the proper amount, type and quality of services? 


1 —_—, What are participants’ experiences of contact with the programme, e.g. how were 
they invited, what kind/how many contacts did they have, what was the 


duration/content of their contact? 


_Do participants understand the nature of the programme, its intention and its 


various elements? 


_Are participants satisfied with their interactions with staff delivering the 


programme, with the procedures, and with the services they receive? 


o _ Are there particular groups within the target population which do not receive the 


programme and why? 


OH  —_, Do participants engage in anticipated or intended follow-up behaviour? 


These questions may need to be answered with a variety of approaches including 
operational research, social research, monitoring, or analysis of administrative data (e.g. 
benefit records) and programme databases. Ideally, different sources of evidence should 
be used to triangulate, verify, substantiate or qualify findings. The main research methods 
are discussed in Section 3. 

ii Can process evaluation be used to look at outcomes of the programme? 

Although process evaluation is not concerned primarily with outcomes, the 
methods used (both quantitative and qualitative) often inform learning about outcomes, 


and may be crucial to the success of an impact evaluation. By and large, information 


collected as part of a process evaluation will help to address the following sorts of 
questions, although it may not always be possible to talk in terms of specific quantities 
(which the impact evaluation is designed to do): 

(| What effect (positive and negative) is the programme perceived to have on 


programme participants (and non-participants)? 


_What effect does the programme appear to have more widely, for example in 


terms of cultural change? 


0 —_-. What factors appear to underpin differing impacts and outcomes? 


_Are there particular sub-groups within the target group which do better or worse 


from the programme and why? 


_Do particular models of delivery or implementation appear to produce better 


outcomes? If so, how and why? 


_Are there groups for which the programme appears to create more sustainable 


outcomes? 

iii When to attempt a measure of the impact of a programme 

In Sectivn 4, we outline some of the various ways in which researchers might attempt to 
measure the impact that a programme has on a relevant oulcome or outcomes. In most 
instances, it is extremely difficult to make an accurate estimate of the programme’s 
impact. To do this, we must measure what would happen — to the relevant outcome or 
outcomes - if the programme were not in place. This is called, measuring the 
counterfactual. For programmes where the expected impact is small, the sample sizes 
needed to carry out impact evaluations are often very large. So, if the counterfactual 
cannot be estimated using administrative data, then a considerable proportion of the 
budget for an evaluation will often be devoted to its estimation. It is important therefore, 
that the need to estimate the counterfactual is carefully assessed before going ahead. 

Asa starting point, the counterfactual should only be estimated when the primary 
outcomes for a policy or programme are expressed in terms of change. For instance a 
policy objective might be written in terms of increasing the number of persons in work, 
or reducing the numbers on low incomes. In these instances the counterfactual is the 


figure from which the increase or reduction is achieved. 


In some instances there is a strong case for assuming a particular counterfactual. 
For example, a new service might be introduced where the main target is to encourage a 
take up rate above a fixed percentage each year. In this instance the counterfactual would 
obviously be zero. A measure of the counterfactual is useful: 
Oo _When a good estimate of ‘added benefit’ or ‘additionality’ is needed for a 


costbenefit analysis 


O _ When there is a need to convince others that a programme or policy is beneficial 


O , When there are alternative policies or methods of implementation that might be 


used and there is a need to establish which is preferable. 


o _ When there is a need to establish whether a policy works better amongst some 


sub-groups than others. 


Nevertheless, the counterfactual will be very difficult to estimate reliably if the 
programme impact is expected to be very small. The problem is particularly acute if the 
programme impact is smaller than variation that occurs naturally over time in the eligible 


population. 


iv Designing an evaluation workable within the context of the programme 
Clearly, evaluation research cannot be designed in a vacuum. Evaluations of both 

the operation and the outcomes of a programme need to be reviewed within context. To a 
large extent, decisions about the rescarch design will be driven by practical issues such as 
available resources, the timescale for results and so forth. Arguably the main factors that 
will determine the evaluation design are the structure and nature of the programme, 
particularly in terms of: whether the programme is to be introduced nationally or as a 
pilot; whether participation in the programme is voluntary or compulsory; and whether 
the programme can be thought of as a single programme or as a complex set of local 
programmes. In addition, the emphasis put on different elements of the evaluation will 
differ depending upon the concerns of those who conceive the evaluation and those who 
ate most interested in the outcomes. Although these factors will drive the main features 
of the design there are a large number of other factors that will determine the details. 


Some of the questions to be addressed are listed below: 


When is the programme to be implemented (immediately or in the future)? If in 


the future, can and should data for evaluation be collected before implementation? 


_ When do the findings from the evaluation need to be available? 


What policy changes are anticipated during the evaluation period? How might the 


changes affect the research? 


_ Are there different models of delivery that need to be evaluated separately? 


For instance, are some people offered a telephone help-line while others are given 


advice face-to-face? 


Are there different sub-groups for which the programme needs to be tested 


separately? For example, do we need to look across different client groups, 
service providers, age groups, people receiving different benefits or benefit levels, 
etc.? 

(] | What geographical areas need to be covered in the evaluation, and what areas 


need to be avoided (perhaps because another intervention is being piloted there)? 


_ What arc the outcomes that we would like the programme to change? Can these 


be prioritised? 


CG ~—-, What impact is the programme likely to have? What is the smallest impact we 
would like to be able lo detect? What is the smallest impact that will he deemed to 
be a success (or failure)? 


O _, What are the factors that will affect the impact? 


_ What are the participation rates likely to be? 


_ What data is currently available that the evaluation can make use of? What extra 


will need to be collected? How will that data be collected? 


_Are monitoring processes established and collecting satisfactory data? [What 
resources are available? This applies to different types of resources, including 


money, staff, time, central Government staff, etc. 


The answers to these questions will largely determine how the evaluation is 


designed. Nevertheless, the various requirements of the research may mean that the 


design process goes through several stages before a satisfactory design is found which 


will address all the necessary questions of the evaluation. 


v Persons to be included in the evaluation 

An important decision that needs to be made during the early stages of the 
development of an evaluation is who are the 'subjects' of the evaluation. That is to say, 
which actors in the process in question need to be studied in order to effectively answer 
the questions set out in the evaluation? The relevant actors can be considered in three 
main groups: 
Primary subjects 
These are the individuals to whom the policy or programme is specifically targeted or 
who are directly affected by the policy or programme. Examples are - 


o _ Eligible/target population 


_ Claimants 


_ Recipients 


Non-recipients (perhaps divided into those who have or have not claimed the 


benefit, eligible non-recipients, potentially eligible non-recipients etc.) 


. Participants 


O Non-participants (who have or have not engaged with the programme) 


_ Leavers (from a benefit or programme) 


Secondary subjects 

These are the individuals who may have a key role in making the policy or programme 
work but are not integrally involved in the development or delivery of the policy or 
programme. For example, in the New Deal programmes, employers are crucial because 
they supply the work opportunities that make the New Deal effective. Representatives of 
these groups may be involved to a lesser or greater degree in the strategic development of 


the policy or programme and these too may be incorporated in the evaluation. Examples 


are - 


, Employers 


_ General Practitioners 


a , Pension managers 


_ Intermediaries such as CAB, advice centres, referral agencies 


_Other subjects indirectly affected by a policy that is not delivered by them, for 


example a hospital emergency wards affected by people being encouraged to visit 

their GP. 
Actors involved in delivery 

One of the key groups usually studied in evaluations are those involved in the 
delivery of new policies or programmes. Furthermore, the new emphasis on 
crossdepartmental working and partnerships, means that additional actors and agencies 
are increasingly involved in the delivery of policies and programmes and so are included 


in evaluation designs. Examples are - 


_ Policy makers/implementers 


0 _ Staff involved directly in delivery e.g. programme managers, personal advisers, 


adjudicators, independent organisations contracted to deliver services 


_Secondary actors involved in delivery e.g. Employment Service staff, Local 


Authority staff and independent contractors. 


ce, Process evaluation 
Almost all large-scale evaluations of government policy will include some 
elements of process evaluation since, arguably, this is the side of the evaluation that 
provides most information on how the policy should be managed or developed in the 
future. In formal terms process evaluation (Scheirer, 1994) "verifies what the program is 


and whether or not it is delivered as intended to the targeted recipients". 


i Uses of a process evaluation 

Process evaluation tends to be carried out independently of service delivery and is 
normally carried out by evaluation specialists. For practical reasons, it is normally 
separate from day to day programme management and monitoring, though data from 
Management Information Systems can play a vital role and be built into the design of the 
programme to allow for evaluation. In some instances a process evaluation may be all 


that is needed for an evaluation. Possible scenarios where this might arise are: 


0 _for a relatively new programme where further development is likely before an 
assessment of the impact of the final model is needed. In this instance the process 
evaluation would be considered as ‘formative’; ‘ 

o _with an established programme which is under-performing or where questions 


have arisen about organisation, delivery, quality, or success; 


with a programme where effectiveness is known or assumed and only the 


implementation and delivery is in question; 


_where impact evaluation is ideally wanted but the size of the programme, its 


expected effect, or the length of time available to allow outcomes to emerge are 

too small to make outcome evaluation possible. 

More typically a process evaluation will be carried out alongside an impact 
evaluation. In these instances the process evaluation data can be used independently of 
the impact evaluation to assess the implementation procedures. But perhaps just as 
importantly, the process evaluation provides very useful contextual data against which 
the results from the impact evaluation will be judged. It is common for instance, for an 
impact evaluation to give one overall figure for the impact of the programme on the 
eligible population in aggregate, but for the sample numbers to be too small to allow for 
reliable estimates on impact on sub-groups to be made. In these instances the process 
evaluation findings can often be used to make judgements (sometimes qualitatively rather 
than quantitatively) about where and amongst who the impact of the programme is 
greatest. 

ii Methods of process evaluation 

Early sections of this paper have discussed the role of process evaluation and the 
answers which it may address (Section 2.1). The issue about who should be included 
within the evaluation has also been discussed (Section 2.5). In this section, we look at the 
variety of research methods available for answering the questions and communicating 
with the relevant actors. It is important to establish what method or methods of data 
collection will be most effective within each evaluation. Each method will of course carry 
with it particular advantages and constraints (as well as cost implications). Often, a 
combination of methods is necessary to answer all the questions posed by an evaluation. 


Evaluations which combine methods successfully are also likely to be more robust, where 


they are able to combine findings pertaining to the same topic from different sources. 
They will also, of course, often be the evaluations that are the most complex to analyse 
and interpret. 

The range of methods might be divided into two broad groups, namely 
monitoring/operational research and social research, the key distinction for our purposes 
here being that monitoring and operational research use data collected for purposes other 
than the evaluation proper, whereas social research methods are usually used to collect 
data specifically for the evaluation. The two methods are described in turn below. 

e. Monitoring and Operational Research 


Monitoring and Operational Research may involve: 


The analysis of administrative data about the eligible population e.g. benefit 


records 


The analysis of data from Management Information Systems e.g. programme 


database 


The collection and analysis of performance measurement data about resources 
(staff and financial), caseloads, etc. 
0 _ Special monitoring exercises e.g. pro formas attached to specific cases to identify 


stages and timing 


_ In-depth area studies 


It is usual for Government departments to use administrative data to monitor the 
progress and, where possible, to estimate the impact of programmes. In addition 
Operational Researchers will often carry out specific modelling tasks, examples of which 


include: 


Analysis of customers’ behaviour, in terms of their interactions with the 


organisation, during and subsequent to their involvement in the programme. This 


involves building models of the process of interaction. 


_ Statistical analysis of data collected from IT systems, or from customer surveys, 


to establish optimal methods for encouraging participation. 
f. Social research 


Social research, used in the context of evaluation, might involve: 


_ Large-scale quantitative surveys 


| 


(| . Qualitative approaches using depth and paired depth interviews or discussion 


groups 
O _ Case studies 
Oo _ Literature reviews, observational studies, diaries and documentary analysis. 


Employing social research methods inevitably adds to the costs of an evaluation since 
they involve primary data collection. ‘hey arc used when it is considered that monitoring 
and administrative data is either incomplete (for instance, if it provided no information 
about employers) or insufficient. Below we describe the three main methods of data 
collection of social research: quantitative surveys; depth interviews and group 
discussions; and case studies. 
g. Quantitative surveys 
The main role of surveys in evaluation is to collect quantitative data on (usually large 
numbers of) the subjects of the evaluation (see Section 2.5), such as participants and local 
employers. Surveys of those involved in the delivery of the programme are much less 
common because of the small numbers involved. Any surveys, especially surveys of the 
eligible population, will usually be designed to answer both process and impact 
questions, and this dual role will largely determine survey design and sample size. Some 
of the conflicts that are likely to arise are described below: 

O _ For process evaluation it may be important for surveys to be ‘representative’ of 
the eligible populations in the areas the programme covers. Impact evaluations are 
more likely to need samples that are comparable with control groups than are 
representative per se. 

lJ For voluntary programmes participants are likely to be of more interest for a 
process evaluation than are non-participants (although some process questions 
will be specifically targeted at non-participants). There may therefore be a strong 
argument for over-sampling participants in any survey. Depending upon the 
design of the impact evaluation this over-sampling may or may not be appropriate 


for the impact estimation. 


_ Similarly, for a process evaluation there is likely to be some value in stratifying 
the survey sample in a way that maximises the possibility for sub-group analysis 


(either in terms of the characteristics of the eligible population, or in terms of the 


type of programme intervention delivered). This may conflict with the needs of 


the impact evaluation. 


_Pre-participation surveys may, for process evaluation purposes, need to cover 


questions about awareness and attitudes towards the programme. Focussing 
attention on the programme may however contaminate the impact evaluation. The 
needs of the process evaluation are likely to drive factors such as interview length 
and mode. For instance, it is likely that a short telephone interview would be 
sufficient in many cases to collect information on the outcomes needed for the 
impact evaluation. In contrast, the demands of a process evaluation are more 
likely to point to a lengthy face-to-face interview. Furthermore, if an aim of the 
process evaluation is to examine how the programme changes over time then this 
may require several surveys rather than just one. In contrast, the need for very 
large sample sizes is likely to be driven by the demands of the impact evaluation. 

Qualitative research: depth interviews and group discussions 

Qualitative research (in particular depth interviews and group discussions) 
provides in depth investigation of the nature of social and organizational behaviours and 
how and why they occur. It is characterised by the use of exploratory and interactive 
methods of data collection that aim to capture the form, complexity or origins of the 
issues being reviewed. Qualitative approaches are considered particularly appropriate for 
process evaluation because they enable researchers to explore, in great detail, the efficacy 
of the organisation and delivery of a programme. These approaches allow evaluators to 
assess the features which are appraised as more or less effective by those directly 
involved in either the design, delivery or receipt of the interventions. In addition, 
qualitative approaches allow for an investigation of the range of factors which can affect 
overall outcomes and provides detailed exploration of the factors underpinning 
participants’ experiences of programmes. 

Qualitative research is very different to survey research and has very different 
uses in evaluation. Whereas surveys provide the hard quantitative estimates needed to 
answer the ‘how many’ questions, but fail on many of the depth questions about ‘why’ 
and ‘how’, qualitative research cannot give quantitative estimates, but does allow many 


of these more depth questions to be addressed. Because of these different focuses, 


| 
| 
| 


qualitative research is often used alongside surveys. Furthermore, because of the small 
number of personnel involved, qualitative research is the main tool used amongst 
programme delivery staff. Most evaluations will, however, also include qualitative 
research amongst participants even when participants are covered by surveys. Qualitative 
process evaluations can involve a range of different methods of data collection. Popular 
tools include individual depth interviews and group discussions. Other approaches which 
might be employed include paired depth interviews, observational work and documentary 
analysis. Individual or paired depth interviews use a topic guide (or interview 
schedule) which lists the key themes and sub-themes to be explored. Individual 
interviews are an ideal forum to explore detailed personal experiences of interventions 
allowing respondents to describe and evaluate their personal experiences of the 
intervention. They are also suited to eliciting personal or sensitive information. Group 
discussions (also called focus groups) usually involve around six to eight respondents, 
with a moderator or two co-moderators. Group discussions provide an appropriate area to 
bring together participants/recipients or service providers to discuss, share and compare 
their experiences of the intervention. The exchanges that occur between respondents can 
highlight common experiences and views, identify differences within the group, and act 
as a stimulus to further thought among respondents. They are also a stimulating 
environment for generating solutions and strategies. However, they are less suitable 
where the evaluation requires exploration of detailed personal accounts such as 
employment histories or detailed experiences of service delivery. 

It is worth noting the special role of /ongitudinal qualitative research in process 
evaluation. Longitudinal elements can help to identify changes to the delivery, practices, 
and organisational management of interventions/services/programmes. In particular 
revisiting a previous sample can help to identify the nature of changes over time to 
attitudes, decision-making , behaviours; the factors influencing the durability of 
outcomes; and provide reflective perspectives on the experiences of different sub-groups 
within the sample. A paper on the use of longitudinal qualitative research in evaluation is 
to be published in the series. 

Examples of the use of qualitative research 


New Deal for Young People (NDYP) 


The evaluation of NDYP included a series of six qualitative studies amongst 
participants to review different stages of the programme. The research objectives were to: 
to provide information about participants’ experiences of the different components of 
New Deal and their appraisal of them; to examine the nature of the impact of the 
programme on participants’ aspirations and employment or educational outcomes; to 
identify reasons for leaving the programme and/or the unemployment register and 
identify the factors associated with different routes of; and, to identify the factors that 
facilitate or might enhance the effective organisation and delivery of New Deal. Because 
both in-depth personal accounts and comparisons between participants were required, the 
study included both depth interviews and group discussions. 

The Educational Maintenance Allowance (EMA) 

EMA (a DfES programme) provides financial incentives to school leavers to 
encourage participation in post-compulsory education. A series of extension pilots are 
being implemented which aim lo provide addilional, or different forms of, financial 
support for students with children, disabled students and students who are homeless. As 
part of the evaluation programme a quantitative survey was carried to establish the 
factors which influence the choices young people make following their post-compulsory 
education (PCE) at 16. A qualitative study followed-up the survey respondents exploring 
in greater depth their decision-making at this time, views, experiences and impact of 
participation in EMA. The key aims of the qualitative study were to: explore attitudes to 
post-compulsory education; identify factors which influence participation in post- 
compulsory education; explore attitudes to and experiences of the EMA scheme; 
investigate the impact of EMA on participation, retention and achievement in post- 
compulsory education; explore the impact of EMA on financial decision-making, 
expenditure and transfers within households; compare and contrast different EMA 
variants; and to identify suggested changes and improvements to EMA. A qualitative 
approach was chosen to provide a detailed understanding of the operation of EMA 
scheme. The study was carried out using depth interviews amongst a purposively selected 
sample of young people and their parents. The design also incorporated a longitudinal 
element to allow the longer-term impacts and outcomes from the scheme to be explored 


as a complement to the Year One process evaluation data. 


i Case studies 

The focus of depth interviews and group discussions is the experience of 
individuals seen from their perspective. Case studies, in contrast, look at individuals or 
organisations from the multiple perspectives of key actors. These perspectives help to 
build a detailed understanding of the experiences and outcomes in a specific case, where 
a case can be an individual client/participant; an area; an office etc. Case studies are 
generally included in an evaluation when the evaluation requires a detailed understanding 
of these multiple perspectives on an intervention. For example a case study might be used 
to explore, in detail, an individual client's experiences of a programme where the roles or 
accounts of different actors are seen to affect the delivery or impact of the programme on 
that individual. Alternatively, case studies may be appropriate when exploring factors 
accounting for different models of delivery where multiple actors are involved in 
designing or implementing an intervention. 
Examples of the use of case studies 
New Deal for Disabled People 
In the evaluation of the New Deal for Disabled People pilot, case studies were used to 
examine the ways in which partnerships for delivery were operating in different pilot 
areas. Each case study involved depth interviews with key actors from different 
organisations within the partnership. 
Education Maintenance Allowance 

EMA (a DfES programme) provides financial incentives to school leavers to 
encourage participation in post-compulsory education. A series of extension pilots are 
being implemented which aim to provide additional, or different forms of, financial 
support for students with children, disabled students and students who are homeless. The 
evaluation of EMA (see Section 3.4.1) involves case studies of individuals. Each case 
study involves the investigation of the circumstances and key issues particular to a single 
individual students’ case by seeking the multiple perspectives of key actors. Each study 
involves a depth interview with the student and depth interviews with nominated key 
support workers, family members or significant others. 


j. Impact evaluations, or methods of estimating the counterfactual 


As we have outlined earlier, the primary aim of an impact evaluation is to 
measure whether a particular programme has achieved its desired outcomes. To do this, 
we measure outcomes with the programme in place and compare then to outcomes 


without the programme (i.e. the counterfactual). Measuring outcomes with the 


programme is relatively straightforward since it is largely a matter of observing what 


happens. Measuring the counterfactual is much more difficult since no direct 
observations can be made. A variety of different methods have been proposed for 
estimating the counterfactual. The feature all the methods have in common is that they 
compare a groups of people involved with the programme — which we term the 
‘intervention’ or ‘treatment’ group — with a group of people who are not involved - the 
‘control’ group. The counterfactual is then estimated as the outcomes for this control 
group. The intervention group will usually be everyone who is eligible to be involved in 
the programme (the ‘eligible population’), or a sub-sample of the eligible population, for 
instance people who are actually participating in the programme. In principle, the control 
group should be people who are, on average, identical to the people in the intervention 
group, with the single exception that they are not involved in the programme. Much of 
the discussion about the relative merits of different evaluation methods is centred around 
the extent to which the control group can be said to truly mirror the intervention group in 
all areas apart from the programme intervention. 

The only design where this assumption of equivalence between the intervention 
and control groups is guaranteed is the ‘randomised trial’ (often referred to as an 
‘experimental design’). This is the design considered first (sub-section 1) in the pages 
that follow. All other designs are referred to as ‘quasi-experimental’. The ones we 
consider here are the matched area comparison design (sub-section 2), the before-after 
study (sub-section 3), the matched comparison group design (subsection 4), and 
difference-in-differences (sub-section 5). A short discussion of the role of cost-benefit 
analysis in impact evaluation is included as section 4.6. 

Sections 4.1 to 4.5 below focus on the theory and logistics of each approach. For 
most designs the basic data collection procedure is the same: the two groups (intervention 
and control) are identified at one point in time and they are then followed up over time. 


After a suitable interval outcomes are measured, using either survey or administrative 


data. The outcomes for the control group give the estimate of the counterfactual; the 
difference in outcomes between the control and intervention groups gives an estimate of 
the programme impact. Before turning to the main impact evaluation methods it is worth 
mentioning one relatively crude method that can be used in conjunction with other 
methods, which is to ask participants in a programme to say what the counterfactual 
would be. If participants can assess what would have happened to them if they had not 
participated, then an estimate of the counterfactual can be derived directly from those 
assessments. As an example of this, on the NDLP prototype, participants who found work 
were asked to say whether or not they would have found that work without the help of the 
personal advisor. Those saying ‘yes’ provide the counterfactual. It is unlikely that in this 
instance the estimate of the counterfactual would be given much credence (unless 
supported by other evidence) since it is probable that the assessment is more a statement 
about how helpful the participant found the personal advisor rather than a genuine 
assessment of the help given. 

However, there may be other advice given by the personal advisor for which the 
participant would be more able to assess the impact of the personal advisor. For example, 
did the personal advisor put you in contact with training providers whom you were not 
previously aware of? Or, did the personal advisor give advice on how lo change your job 
search activities that you followed? A very useful role for this general approach would be 
to establish, once the overall programme impact has been established, whether or not a 
programme appeared to work better for some sub-groups than others. 

It should be noted that the main objectives of each design discussed below is to 
estimate the direct impact of the programme on the population eligible for the 
programme. It is, however, perfectly reasonable to expect that the introduction of labour 
market programmes that focus on particular populations will have an impact, often 
negative, but sometimes positive, on other populations as well. 

Evaluation designs rarely include the ability to measure these displacement or 
substitution effects directly, primarily because the effects will usually be thinly dispersed 
which makes detection of the effects very difficult. In designing an evaluation some 
consideration should be given to whether or not this omission is important: if it is then the 


evaluation may have to be broadened. The implications of this on the evaluation design 


are beyond the scope of this paper, but are to be covered in the forthcoming paper in the 
series on impact evaluation. The discussion that follows on each design is necessarily 
brief. 

Randomised trials — an experimental design 

The main features of a randomised trial 

Ina randomised trial the eligible population are assigned at random either to an 
intervention group or to a control group. The control group are denied the programme 
and are treated, as far as is possible, as if the programme did not exist. The control group 
usually receive existing services. 

There are two main types of randomised trial: randomisation of individuals within 
areas; and randomisation of areas. In the latter whole areas are either assigned to the 
intervention or control group. 

The designs’ strengths and weaknesses 

The randomised trial is considered the ‘gold standard’ for evaluation. However, il 
is not always used in practice. This is because of practical difficullies in implementation 
rather than because some other design is more powerful or reliable. The main strength of 
the randomised trial or ‘experiment’, compared to quasiexperimental designs, is that the 
only two differences between the intervention and control group are ‘random differences’ 
and ‘the influence of the programme’. Systematic differences, such as differences in 
motivation between the members of the intervention and control groups, can be ruled out. 
A further advantage is that there is no necessity to do a ‘before-programme’ study to 
check that the intervention and control groups are alike. (See Section 4.5 on difference- 
in-differences.) 

Countering these advantages, are the difficulties (and sometimes impossibilities) 
in allocating people randomly within a particular area to an intervention or control group. 
Firstly, it often means running two administrative systems within a single area. Secondly, 
for voluntary programmes, there is an ethical difficulty in advertising programmes that 
some people will be denied entry to if they apply. In some cases, such as whole 
community interventions, such as a Health Action Zone, it is impossible to randomly 


allocate people in the same area to an intervention or control group. 


Randomly allocating areas, rather than people within areas, to intervention or 
control groups is typically ruled out because of the large number of areas that would be 
needed. The minimum sample size of areas would be about 20 in the intervention group 
and 20 in the control group although in some instances the numbers needed may be much 
greater. Also, compared to a ‘within-area randomisation’ the sample size of people within 
the study would also have to be significantly larger because if the need to allow for 
between-area sampling variation. This has cost implications if the data on outcomes are 
to be collected via surveys. 

Although the results from randomised trials have a credibility that other 
evaluation designs do not, the estimate of the programme effect may be biased if any of 


the following occur: 


_if some of those control group do actually participate in the programme; 


_if surveys to measure outcomes get differential non-response in one or other of 


the groups; 


_if being denied access to a programme makes those in the control group act 


differently to how they would act in the total absence of the programme; 


0 _if those randomised to the programme affect the behaviour or altitudes of those in 
the control group (which might happen if members of the two groups know one 


another);or 


_ if the trial affects the way the programme is implemented. 

If the effect of the programme is expected to be small and/or participation rates 
are low, a randomised trial of the eligible population will need a large sample size. One 
possible exception to this rule is if allocation to the intervention and control groups can 
be done at the point of participation. 

Practical issues 

Most of the practical issues that arise in randomised trial designs occur only if 
people are divided into intervention and control groups within their arca, rather than if 


whole areas are divided into intervention or contro] groups. The main issues are: 


o _Very good management systems need to be put into place to ensure that local 


administrators know who is in the intervention and who is in the control group. 


Systems are needed to monitor, and help avoid, programme participation by any 
of the control group. 

Oo _It is not always immediately apparent who is eligible to be included in the 
randomisation, and who is not. For voluntary programmes, we could decide to 
include the whole target population, or only those expressing an intention to 
participate, or those actively trying to participate (or, in principle, at any other 
point along the continuum between ‘eligibility’ and ‘active participation’). The 
closer we get to ‘active participation’, the smaller the trial can be. However, the 
costs, practical and ethical difficulties of identifying those reaching a particular 
point on the line may offset any such advantage. Furthermore, the impact of the 
programme on those excluded from the randomisation cannot be known. (So if 
randomisation is done at the point of participation, the impact of the programme 
on non-participants will not be measured.) 

Oo The size of the control group within each area has to be kept relatively small if the 
programme being offered to the intervention group is to be a reasonable 
approximation to the programme that would be offered to everybody. This will 


tend to increase the number of areas that the trial needs to be run in. 


_It may be considered unethical to extend a trial beyond the point at which a 


positive benefit of the programme has been demonstrated. But because at the end 

of a trial the control group will usually be free to participate, the trial then gives 

no evidence on the long-term impacts of programmes. Longer-term benefits tend 
to be estimated using quasi-experimental methods. 

Examples of randomised trials 

Intensive Gateway Trailblazers (IGTs) 

For all IGTs clients were randomly allocated to either an IGT group or to a 
control group. The randomisation was done using NINO digits: clients with an odd NINO 
were assigned to the IGT group; those with an even NINO were assigned to the control 
group. 

Fully Fledged Employment Zones 
In the impact evaluation of the Fully Fledged Employment Zones two methods 


were used to estimate the counterfactual. Within four of the zones individuals who were 


eligible for the zone were randomised to one of two groups: intervention and control. As 
with the AGTs the randomisation was done using NINOs. The remaining 11 zones were 
evaluated using a matched area comparison design (see Section 4.2). In the 11 zones all 
those eligible for the zone were included in the programme, and these 11 areas were 
matched to comparison arcas with similar labour market characteristics. For both 
methods data on outcomes was derived from administrative sources, namely MI, JUVOS, 
and LMS. 
NDLP Prototype 

The impact evaluation of the NDLP prototype was designed as a matched area 
comparison (see Section 4.2) but within each area the invitations to join the programme 
were sent out over a period of time and in essentially random order, the date the 
invitation was sent being determined by NINOs. For the early part of the prototype this 
mimics a randomised trial, those invited early being the intervention group and those 
invited later being the control group. In practice some of the randomisation was lost 
because members of the ‘control’ group were allowed to self-refer onto the programme. 
Nevertheless, the results from this ‘natural’ experiment were considered by some to 
provide a more robust estimate of the counterfactual than the matched area comparison. 
k. Matched area comparison design 
The main features of a matched area comparison design 

In this design, a new programme or policy is piloted within a small number of 
selected pilot areas (typically about 10). These areas are then matched to a set of ‘control’ 
areas, usually, although not always, on a one-to-one basis. The control areas are carefully 
selected to match the pilot areas in terms of their labour market characteristics. The 
population eligible for the programme is identified within each selected area — both pilot 
and control. Either all the eligible population or just a sample is followed up. Then, after 
a suitable interval, outcomes are recorded. Differences in outcomes between the pilot 
areas and the control areas are attributed to the programme. Typically this is after 
controlling for any major differences between the eligible populations of the areas. 
The designs’ strengths and weaknesses 


The primary strength of the matched area comparison is the ease with which it can 


be implemented. It avoids most of the problems of randomly allocating people to 
intervention and control within areas. Only one administrative system per area is needed, 
and no individuals within pilot areas are denied access to the programme. For this reason, 
matched area comparisons are usually considered more ethically acceptable than 
randomised trials of individuals within areas. 

The main weakness of the design is that the results from it are often very difficult 
to interpret. The difficulty arises because of the main feature of the design, namely that 
the control group members all live in different areas to the pilot group members. This 
means that any observed difference between the pilot and control groups might, 


potentially, be attributable to three things - 


_ Differences in the labour markets in the pilot and control areas; 


Oo _Differences in the eligible populations of the areas (these might include 
differences in factors such as: motivation to find work; ability to look for work, 


and qualifications for work). 


O The programme itself. 

To estimate the impact of the programme itself, we have to control for both of the 
first two factors. In principle differences in labour markets are controlled for by carefully 
matching the pilot and control areas at the selection stage. But, no matter how carefully 
this matching is done, residual differences are likely to remain. For programmes targeting 
special populations who experience the labour market in non-standard ways there is the 
additional difticulty of being uncertain about what aspects of the labour market it is 
important to match on. Furthermore, there is the problem that a good match at the start of 
the programme may prove to be a much poorer match at the time outcomes are measured. 
Changes in specific areas may occur over time that make them different at the end of the 
study. For these reasons ‘mis-matching’ has usually to be controlled for at the analysis 
stage. With a small number of areas this can only be done crudely. 

Differences between the people eligible in the pilot and control areas, such as 
personal circumstances (eg. child care responsibilities), previous work experience, 
educational qualifications, and motivation to find work, are usually controlled for at the 
analysis stage using statistical modelling. If the analysis is based on administrative data, 


then the ability to control for these factors is very limited. There is more potential with 


survey data, but the estimates of the programme effect may be sensitive to the adequacy 
of the models used and the reliability of the data. 

The problem of controlling for these factors is particularly acute when the 
programme effect is expected to be small. As an example, the NDLP prototype estimated 
that the number of lone parents finding work was about two percentage points higher in 
the prototype areas than in the control arcas. It is difficult to feel confident that this 
difference is entirely attributable to the programme and not, at least in part, attributable to 
labour market and/or population differences between the prototype and control areas, for 
which the statistical modelling could only partially control. 

Appropriate uses of a matched area comparison design 

This design is appropriate when the expected impact of the programme is 
considerably greater than any variation which we might expect between the areas if there 
was no programme in place (and after controlling for labour market and personal 
characteristics). This variation will rarely be known exactly, but it will often be possible 
to make an estimate using a few years of historical administrative data (with ‘leaving 
benefits’ as one possible outcome measure). 

In estimating the expected impact of the programme — the ‘expected programme 
effect’ - account needs to be taken of the programme participation rate. A high impact on 
participants, but a low participation rate, will lead to a low overall programme impact. 
For matched area comparison designs, which compare eligible populations between areas 
rather than compare participants, low participation rates can make an impact almost 
impossible to detect. (Other designs, such as the matched comparison group design (see 
Section 4.4), measure the effect on participants only and so low participation rates are 
less problematic.) 

Improvements on the basic matched area comparison design 


The basic design might, in some circumstances, be improved by: 


Increasing the number of areas in the study. This might be achieved by 
implementing a pilot programme within areas smaller than BA districts, but using more 
BA districts. 

Oo _Incorporating a ‘before-programme’ estimate within both the pilot and the control 


areas. This might be achieved using administrative data or by collecting retrospective 


a 


work histories from sampled members of the eligible population. (Essentially this 
approach means extending the design to incorporate a difference-in-differences estimate 
— see sub-section 4.5). 

Examples of a matched area comparison design 

NDLP Prototype 

For the evaluation of the NDLP Prototype the eight prototype areas were matched 
to six ‘control’ areas. Outcomes were measured in terms of exits from benefits (which 
could be monitored using administrative data, and on the whole of the eligible 
population), and in terms of entry to work (which was monitored using survey data on 
sub-samples of the eligible population). In the event no significant differences were 
found between the prototype and control areas and it is likely that this was because of the 
relatively low participation rate (at about 20%): an estimated 10% additional employment 
amongst participants equated to just a 2% difference in entries to employment between 
the eligible populations of the prototype and control arcas. Since the matched area 
comparisons findings were largely consistent with those of the NINO comparison (see the 
example in Section 4.1) it was possible to attribute this 2% difference to the programme; 
without the NINO comparison the 2% might just have easily have been attributed to a 
residual difference between the prototype and control areas. 

Earnings Top-Up (ETU) 

In the evaluation of the Earnings Top-up, the benefit was introduced in eight pilot 
areas. ‘hese eight areas were matched with four control areas, which were very similar in 
their characteristics with the exception of the benefit introduction. Within the twelve 
study areas, surveys were conducted of employers, low paid workers in work and the 
medium-term unemployed. Outcomes were measured in terms of net income and entry 
into work. The evaluation also incorporated a longitudinal element, resulting in a 
differences-in-differences design, discussed below in Section 4.5. 

ONE 

The ONE evaluation also uses the matched area comparison approach. ONE has 

been introduced into twelve pilot areas, and these twelve areas have been matched to 


control areas (twelve in total, but not using a one-to-one match), where ONE has not been 


implemented. The outcomes of the participants in the pilot areas will be compared with 
those for similar people from other, comparable areas. 

1 Before-after study 

The main features of a before-after study 

Before and after measurements can be incorporated into most evaluation designs. 
What we refer to here as a ‘before-after study’ is just one particular instance of their use. 
Ina standard before-after study, outcomes will be measured on the population eligible for 
a programme both before the programme is implemented and after. The difference 
between the before and after measurements is taken to be the impact of the policy. (In 
this instance, the ‘before’ — or ‘baseline’ — measurements act as the control 
measurements.) Typically outcomes are measured at just one point in time before 
programme implementation and at one point in time after implementation. But this basic 
design is considerably strengthened if the number of measurement occasions is increased 
hoth before and after. Before-after studies are primarily used in instances where a policy 
is to be implemented nationally without a pilot stage. 

The strengths and weaknesses of a before-after study 

The key strength of the before-after study is that it is possible, in theory, to 
implement a policy nationally and yet still obtain a measure of the impact that policy has. 
In practice, the design has considerable weaknesses that make its use very problematic. 
The main weakness of the design is that change brought about by the policy cannot be 
separated out from change that would have happened anyway (i.e. ‘natural change’ or 
change brought about the introduction of other policies at about the same time). This is 
particularly problematic if the policy is expected to have a relatively small impact, the 
worst case being where the expected change due to the policy is smaller than the change 
that happens ‘naturally’ from year to year. 

The design can be strengthened quite considerably if the time series is extended to 
several years (or periods) before the implementation of the policy and several years after 
the policy is implemented. It then becomes possible to look for an ‘interruption’ or ‘shift’ 
in the time series at the time the policy is introduced and to check that the shift is 
sustained over time. However, although this is a relatively powerful approach, the strong 


data requirements means that it is usually only possible to use administrative data or other 


standard datasets (such as large repeated government surveys). This limits the outcome 
variables that can be used. Furthermore, the approach means that results on the success of 
the policy will only be available some considerable time after the programme is 
implemented. 

Design issues 

The before and after measurements can be taken on different cross-sections of the 
People eligible or by taking repeated measurements on the same people within the 
eligible population. One variation on this is to select a sample once a policy has been 
implemented, and then to collect data on that sample retrospectively. Although in 
instances where the eligible population is reasonably constant over time, it is often 
preferable to take repeat measures from the same sample - in that it makes measures of 
change more precise - there is a danger that change over time will include not only 
‘natural change’ and change due to the programme, but also include a ‘age-effect’ (Le. an 
cffect due to the fact that that the sample members are older ‘after’ than they were 
‘before’). Repeat measures data should consequently be used with extreme caution. 
Appropriate use of the before-after designs 

Before-after and time series approaches are probably of most use as a supplement 
to other more formal evaluation methods. In particular, the administrative data collected 
by government departments allows for long-term time series analyses to be carried out 
relatively easily and cheaply. The results from these analyses play a very useful role in 
validating and confirming the conclusions from the short-term intensive evaluations 
carried out by independent researchers. 

Examples of before-after designs 
Jobseekers’ Allowance (JSA) 

To evaluate the impact of the introduction of JSA two samples of benefit 
claimants were selected, one before the introduction of JSA and one after. Each of the 
two samples was selected so as to be representative of the unemployed claimants in 
Britain at the time of selection. The sample members were all interviewed twice: at the 
time they were selected and after six months when data on outcomes was collected, the 
primary outcome being entry into paid work. The difference in outcomes between the two 


samples was interpreted as the impact of the programme. In practice interpreting the 


change in this way was problematic because of macro economic change that occurred 
over the interval. 

m. Matched comparison group design 

Main features of a matched comparison group design? 

In a matched comparison group design, the people eligible for a programme (the 
eligible population) are divided into two self-selecting groups - participants and non- 
participants of the programme. An intervention group is then selected from the 
population of participants and a control group is selected from the population of non- 
participants. The controls are selected so that they ‘match’ the participants. 

The usual approach is to match each selected participant uniquely to a 
nonparticipant. If the matching is done well, the non-participant controls will be identical 
to their matched participants in all relevant respects with the single exception that the 
participants participate and the controls do not. ‘Relevant respects’ for matching are 
factors that are associated with participation and with the outcome(s) of interest. For 
example if the outcome of interest is ‘finding work’ and participants are 
disproportionately self-motivated well-qualified people whose personal circumstances 
make working a feasible option, then the non-participant controls should have the same 
characteristics. 

Once matches are found, both participants and their matches are followed-up over 
time and outcomes recorded. The difference in outcomes between the participant and 
control groups gives an estimate of the impact or effect of the programme. 

The strengths and weaknesses of a matched comparison group design 


The matched comparison design has a number of strengths: 


_It is a very useful evaluation method in the instance where a programme has been 


introduced nationally (if done well it should be less biased than a simple before- 


after study). 


There are few, if any, ethical difficulties because nobody is denied the 


programme. 


_ Unlike the matched area comparison where the intervention group is everyone 


who is eligible for the programme, for whom overall programme effects are likely 


to be small, the matched comparison group design concentrates on participants 


(where arguably all of the programme impact is concentrated). As a consequence 

the sample sizes needed for a matched comparison design are often considerably 

smaller than those needed for a matched area comparison. If outcomes are to be 

collected using survey data this can significantly reduce the costs of evaluation. 

On the down side, the design is very dependent upon the matching procedure. 
Any inadequacies in this can introduce bias into the estimates of programme effects. As 
an illustration of the problem, let us suppose that NDDP participants are matched to non- 
participants on age, sex, disability, and duration on benefits. But suppose that, within an 
age, sex, disability, and duration combination, participants are more likely to be actively 
seeking work than are the nonparticipants. If this is the case, even without the 
programme, the participants will be more likely to find work than their matched non- 
participants, and this is the case irrespective of their participation. In this scenario the 
effect of participation (i.e. the programme effect) is likely to be over-exaggerated. 
Practical issues 

To do the matching well means that all of the relevant matching variahles have to 
be identified, plus reliable data needs to be available on all of the relevant variables for at 
least a sample of parlicipants and non-participants. Furthermore, if some of the relevant 
variables are likely to change with participation (one example might be ‘confidence about 
finding work’), and the information on these variables cannot be collected 
retrospectively, then the information will need to be collected prior to participation. For 
programmes with low participation rates this may involve a very large data collection 
exercise amongst the eligible population. The actual procedures for matching are fairly 
technical, a number of options being available. The method that is most likely to be used 
at present is ‘propensity score matching’. Propensity score matching will be the subject of 
a separate paper in the methodological series but, in broad terms, the method works by 
fitting a logistic regression model of the probability of participation using, as predictors, 
factors thought to impact on both participation and outcomes. Participants are then 
matched to non-participants on the basis of this modelled probability or propensity score. 
The method is an attempt to mimic randomisation at the point of participation, the 
principle being that the participants and their matched nonparticipants are just as likely as 


one another to participate. In each matched pair the decision on which of the two 


participates is treated as essentially a random process. A matched comparison group 
design is only likely to be feasible for programmes with fairly low take-up rates since 
only then will there be an adequate pool of nonparticipants from which adequate matches 
can be found. 

Examples of the matched comparison group design 

NDLP National 

The evaluation of the NDLP National is using a fairly sophisticated version of the 
matched comparison design. A very large (65,000) sample of eligible lone parents who 
have not participated in the programme were sent a postal questionnaire which collected 
information on their characteristics, job experience, qualifications and attitudes towards 
work. Members of the sample who participated within a short fixed period formed the 
‘intervention group’; each of these was matched to another member of the sample who 
did not participate, the matching being done using a propensity score modelled using the 
responses to the postal questionnaire. After a suitable period of time the participants and 
their matched controls are to be interviewed to establish whether or not they have entered 
work, 

Training for Work (TfW) 

A simpler version of the NDLP National design was used to evaluate TfW. In this 
instance a random sample of people leaving TfW were matched to non-participants, the 
matching being based on variables available from administrative systems. The main 
distinction between the NDLP National and TfW evaluations is in the much richer 
material from which matches can be made in the former. 

n. Difference-in-differences 
The main features of the difference-in-differences design? 

The difference-in-differences approach is more appropriately considered as a 
method of estimation rather than as a design alternative in its own right. The approach 
can be combined with all of the preceding designs discussed, with the exception of the 
straightforward before-after design. In a difference-in-differences approach, two groups 
are compared both before and after a programme or policy is implemented. Typically the 
two groups will be participants and non-participants from the same eligible population, 


but the two groups could be the eligible population and some other population (which 


might be the eligible population from a control area). In all cases, one group represents 
the ‘intervention group’ and the second is the ‘control group’. The idea behind the 
approach is that two measures of:change over time (i.e. ‘differences’) are calculated - one 
for the intervention group and one for the control group. The difference for the control 
group gives an estimate of the change over time that would have happened if the 
programme had not been introduced (i.e. it measures ‘natural’ change). The difference for 
the intervention group is a measure of this ‘natural’ change plus change due to the 
introduction of the programme. Subtracting the difference for the control group from the 
difference for the intervention group gives an estimate of the change due to the 
introduction of the programme or policy. There is an implicit assumption that the size of 
the ‘natural change’ is the same for the control and intervention groups. 
Illustration of the difference-in-differences approach 

Consider a voluntary programme where the main objective is to help people into 
work. One potential measure of success would be a higher than average time spent in 
work with the programme than without the programme. A difference-in-differences 
approach might be to divide the eligible population into two groups: participants and non- 
participants. A sample would be taken from hoth of these groups (possibly with matching 
on administrative data variables such as age, sex, length of time of benefits etc.), Data on 
work histories would then be collected from both samples and Lor (wo periods: a fixed 
period before the start of the programme, and a fixed period after the start of the 
programme. (This data might be collected using administrative records, if available, or by 
survey.) For all sample members two figures would be estimated: e.g. percentage of time 
spent in work in the period before the start of the programme, and the percentage of time 
spent in work after the start of the programme. The difference between these two figures 
would then be calculated giving an estimate of change per person. The average of these 
individual differences would be calculated per group. The difference between these two 
averages would be the estimate of the programme impact. In practice the analysis would 
be more sophisticated than this, with, perhaps, the differences being modelled. But the 
principle remains the same. 


The strengths and weaknesses relative to other designs 


The difference-in-differences .approach will, in most instances, give a better 
estimate of a programme effect (i.e. additionality) than the simple before-after design, 
since it makes some attempt to subtract out ‘natural’ change over time. The difference-in- 
differences approach can quite easily be combined with a randomised trial, although it 
is usually unnecessary to do so. This is because the difference-in-differences estimate is, 
algebraically, the same as the difference between the intervention and control group after 
the implementation of the policy minus the difference between the groups before the 
intervention of the policy. In other words, differences between the two groups before the 
start of the policy are being controlled for. In a randomised trial there should be no such 
difference so the extra control is unnecessary. 

The difference-in-differences design can be combined quite naturally with a 
matched area comparison design and should somewhat improve that design. In this 
instance the intervention group would be the cligible population within the pilot arcas and 
the control group would be the eligible population within the matched control areas. In 
both areas, an estimate of before-after change would be made (using either administrative 
or survey data) and the difference-in-differences estimated. In instances where the 
control group is to be selected from eligible nonparticipants, the choice will often be 
between a difference-in-differences approach (along the lines of the illustration above) 
and a matched comparison design. The choice between the two depends on the extent to 
which it is anticipated that the design will remove self-selection bias, with the most likely 
bias for voluntary active labour market policies being that participants are at a point 
where they actively wish to find work. 

Of the two designs the straightforward difference-in-differences approach is 
probably the weaker, since the implicit assumption that ‘natural change over time’ in the 
numbers with positive outcomes is the same for participants and nonparticipants is very 
unlikely to hold if there is self-selection bias (because the participants are a group who 
are likely to experience greater change than nonparticipants, with or without the 
programme). 

However, if the matching for a matched comparison design can only be done 
using administrative variables, then using a matched difference-in-differences approach 


should be an improvement on the straightforward matched comparison group approach. 


In other words, the two approaches should be combined rather than one design being 
chosen over the other. In the instance where more detailed matching is possible, the 
difference-indifferences approach is likely to be unnecessary, since the baseline 
differences between the control and intervention groups should be small. 

Another use of difference-in-differences 

One variation of the approach — which is useful in some instances - would be to 
use the whole of the eligible population as the intervention group and another, similar, 
but non-eligible population as the control group. For example, the intervention group for 
NDLP is lone parents on Income Support, and the control group might be young 
unemployed women without children within the same area. Under the (strong) 
assumption that these two groups are seeking the same jobs and experience the same 
local economic changes, a difference-in-differences approach might be used to identify 
an NDLP effect. In practice, this approach is usually considered unworkable because a 
conirol group who experience the labour market in the same way as the intervention 
group is impossible to find. 

Examples of the difference -in differences approach 
Earnings Top-Up (ETU) 

‘The impact evaluation of ETU, the main design of which is described in Section 
4.2, incorporates a simple difference-in-differences estimator. In both the pilot and the 
vompurison areas, udministralive dala on outcomes was collected both before and after 
the introduction of the pilots. Change over time will be estimated for both types of area, 
and the impact of the programme will then be measured by comparing the change in the 
pilot areas with the change in the control areas. 

New Deal for Young People (NDYP) 

The impact of the NDYP on youth unemployment was estimated by comparing 
change in employment rates for young people before and after NDYP was introduced 
with change over the same period for a slightly older age-group (who were assumed not 
to be affected by NDYP). 

0. ' Cost-benefit analysis 
Although not a design approach in itself, the ability to carry out cost-benefit 


analysis is important within many impact evaluation designs. Cost-benefit analysis 


attempts to identify all the costs and benefits arising from a programme to provide an 
overall assessment of its impact. Comparisons can be made to determine if the benefits 
outweigh the costs and if the benefits net of costs exceed those of alternative schemes. 
Cost-benefit analysis can, therefore, inform decisions on whether to embark upon or 
continue a policy or programme, and choices between alternative programmes. 

Assessing costs and benefits requires firstly identifying the effects or impact of a 
policy or programme (which is essentially where the impact evaluation comes in), and 
then valuing these effects. The results are typically written in monetary values which 
means that additional positive outcomes (such as additional entries to work) attributable 
to a programme have each to be assigned a cost. Placing values on outcomes is clearly 
problematic: in particular the value attributed is likely to differ depending upon the 
perspective of the agents doing the costbenefit analysis. Furthermore, a thorough cost- 
benefit analysis will need to recognise that benefits have different values depending upon 
who receives the benefit. However the data requirements for this level of sensitivity will 
often prove prohibitive. 

By way of illustration, the ONE cost-benefit evaluation will attempt to assess the 
social security savings, taxation gains and wider economic benefits of getting clients into 
work, set against the costs of the programme. This requires an estimate of the additional 
numbers of clients leaving benefit for work as a result of ONE (i.e. the impact of ONE). 
Wider social effects such as those upon crime and health will not be included in the cost- 
benefit analysis because of the difficulty of measuring any ONE impact in these areas. As 
a second example, which illustrates the sensitivity of the findings, cost-benefit analysis of 
the New Deal for Lone Parents (NDLP) suggested that the prototype programme resulted 
in economic returns which were slightly less than the cost of the prototype. It was 
estimated that 20 per cent of the jobs gained by lone parents was as a result of the 
programme intervention. If that figure had been three percentage points higher — namely 


23 per cent — the programme would have had economic benefits equal to its costs. 


p- Exercises: 
1. Please tick mark true or false: 
a. Evaluation designs are same across the subject areas. 
(T) ) 
Ans: (T) 
b. Evaluation designs are same for formative and summative evaluation 
(T) (F) 
Ans: (T) 
c. Evaluation designs are same for process and impact study 
(T) (F) 
Ans: (F) 
d. Clarity of research questions depends on the clarity of the policy 
objectives. 
(T) (F) 
Ans: (T) 
2. Study the research design of mid day meal conducted by the Centre for Equity 
Studies (New Delhi) given at annexure.-1. 
Or 
The study conducted by Collaborative Research and Dissemination CORD, New 
Delhi given in Annexure -2 
Reflect upon its objectives, research design, tools and findings with respect to impact 


evaluation study parameters. 


q. Acknowledgement 


This Unit is based on the paper published by National Centre for Social Research 
Department for Work and Pensions by Susan Purdon, Carli Lessof, Kandy Woodfield and 


Caroline Bryson 


r. References: 


http://www.dwp.gov.uk/asd/asd5/WP2.pdf 


q- Glossary 


The eligible 


population 


Those members of the population who are in scope for the 
programme. For most evaluations the eligible population will be 
defined in terms of a period of time (e.g. lone parents claiming 
Income Support between Oct 98 and Dec ’98). A distinction might 
be made between stock and flow. For voluntary programmes the 
eligible population is usually defined as those eligible to participate 
rather than those who do participate. Some evaluations may, 
however, concentrate on participants, in which case it would be 
legitimate to refer to participants as the ‘eligible population for the 


evaluation’. 


The 


counterfactual 


Also called the base case. It is defined as the number of positive 
outcomes that would have been observed amongst the eligible 
population if the programme was not in place. In most evaluations 
the counterfactual will be measured (with varying degrees of 
accuracy) using a control group who are not in receipt of the 


programme. 


| Deadweight 


The numbers or proportion of the eligible population who would have 
achieved a positive outcome in the absence of the programme. For 
compulsory programmes this will be the same as the counterfactual; 


for voluntary programmes deadweight is often defined for 


participants only. Ie. The numbers or proportion of participants who 
would have achieved a positive outcome in the absence of the 


programme. 


Additionality/pro 


gramme effect 


Number of additional positive outcomes that the programme creates. 
It equals the number of positive outcomes achieved with the 
programme minus the counterfactual. It is a measure of the 


programme effect or impact. 


Displacement/su 


bstitution 


| The change in the number of positive outcomes amongst other non- 
eligible populations as a result of the programme. The estimation of 
displacement is, in most instances, extremely difficult, and most 
evaluations cannot do so with any degree of accuracy. Where an 
estimate is made it is usually through the analysis of administrative 


data. 


Intervention and 


control groups 


‘he intervention group (sometimes called the treatment group) is the 
group in the study who are in scope for the programme. The control 
sroup is the group in the study who are excluded from the 
programme. For some evaluation designs (notably, the matched 
comparison group design), the intervention group is selected trom 


participants and the control group from non-participants. 


Fomative and 
summative 


evaluation 


Formative evaluation is evaluation undertaken to provide information 
that will be used to improve a programme; summative evaluation is 
evaluation used to form a summary judgement about how a 


programme operated. 
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Towards more benefits from Delhi’s midday meal scheme 


1, Background to the study 


Genesis of current midday meal programme in India 


School-feeding programmes in India are most commonly associated with Tamil Nadu where such 
programmes have been in operation since 1956, and became really important after the Chief Minister of 
Tamil Nadu, M.G. Ramachandran expanded their coverage rapidly in 1982 and after.’ The success of Tamil 
Nadu's noon-meal programme* and the comfortable position with regard to foodstocks in the country led to 
the setting up of a National Programme of Nutritional Support to Primary Education (NPNSPE) in 1995; all 
children in government, local-body and government-aided primary schools were to be provided with a 
cooked meal / processed food.” The objective was to boost universalisation of primary education by 
increasing enrolment, retention and attendance, and simultaneously imp act the nutritional status of children 
in the 6-10 age group. 


Unfortunately most states did not follow Tamil Nadu’s example of providing a cooked midday meal to its 
primary school children, and chose instead to provide 3 kgs of foodgrain to children enrolled in primary 
school, with the caveat that they must have 80% attendance in school, In practice, this did not happen and 
while the dry ration scheme succeeded in pushing up enrolment, it had little impact on improving 
attendance, and impacting retention levels. 


The drive to actually provide cooked midday meals in all government and government-assisted primary 
schools came about in response to a public interest litigation on the right to food, initiated by the People’s 
Union for Civil Liberties (PUCL, Rajasthan) in 2001. The context was overwhelmingly convincing — 50 
million tonnes of foodgrain lying idle in FCI godowns coexisting with widespread hunger in the country 
particularly in drought-affected Rajasthan and Orissa. The Supreme Court passed an order directing all 
States to start providing cooked midday meals within six months. The meal was to have a minimum content 
of 300 calories and 8-12 grams of protein each day of school and provided for a minimum of 200 days in 

the year. 


Not all states were responsive to the Supreme Court’s order of 2001. Even by May 2002, some states had 
made little progress, and in May 2003. the Supreme Court asked the laggard states’ to implement the 
scheme in at least 25% of the districts which were poor. By March 2004, 14 states (including Delhi) were 
providing cooked midday meals to all primary school children, 9 states were implementing the scheme 
partially and 4 states were distributing foodgrain. 


The most commonly used excuse was lack of resources. While the central government provided the 
foodgrain and part of the transportation and conversion costs, states had to bear some of these costs 
themselves. This lack of resources reflects the priorities of the politicians. According to one expert, 
“Politicians and bureaucrats tend to be lukewarm about the free mid-day meal programme because there are 


> See Rajivan (2003) for a comprehensive account of the history of school feeding programmes in Tamil 
Nadu. 

4 See Babu and Hallam (1989) cited in Ramachandran (2003) who did an evaluation of the Tamil Nadu 
midday meal scheme in 1984 and found a highly significant increase in school enrolment due to school 
nutrition. 

> Under this programme the HRD Ministry in the central government allocates free foodgrains for primary 
school children at the rate of 100 gm per child for ten months in the year. 

°See SC order, 28 Nov. 2001, 

7 Bihar, UP and Jharkhand in particular. 


very few rent-seeking opportunities in such low-budget schemes. »8 Another reason for such lukewarm 
response from bureaucrats could be the difficulties Set mpleienyne such a massive scheme at a level 

where the meals provided have a minimum quality” The Supreme Court’s specification about giving 
preference to dalit cooks has met with opposition by certain high caste families.'® The programme has also 
met with stiff resistance in some states where the politicians have belonged to a regional party or the main 
Opposition party (the Congress), and been unwilling to put out the resources for a programme for which the 
party at the Centre (the BJP upto 2004) was likely to get all the credit,! 


A right to food: the contribution of midday meals 


In public opinion, the midday meal scheme is generally seen as a populist measure, as just another welfare 
scheme through which some benefits may trickle down to the poor, while a considerable amount is 
siphoned off by functionaries at all levels. 3 Increasingly there are activists and researchers who are trying | 
to draw attention to the lack of food secur ity,! 3 and mounting an ongoing Campaign for the Right to Food.!* 
Linked to this have been the earlier campaign to make education a fundamental right and the more recent 
campaign to guarantee employment to landless labourers. 


While midday meals are a step in the right direction in terms of improving child health and nutrition, they 
are unlikely to be sufficient to make a dent in the problem, In arecent study '* which compared children in 
the 6-10 age group across 5 countries, India fared the worst even though the children were from the better 
off districts of Baroda, Ahmedabad and Rajkot in Gujarat. The children were undernourished, underweight 
and full of infection in spite of the children getting midday meals. A recent review of school feeding 
projects also indicates that the evidence that school feeding programmes are able to improve micronutrient 
status or overall health and nutrition of the child is weak.'® This is not an argument against the need for the 
school meal; only that the situation with regard to health and nutrition is drastic and not as easily improved 
as sometimes indicated. In the Supreme Court orders of 28 Sept. 2001, the States were asked to implement 
fully eight different schemes on food security.” Clearly, the contribution of the midday meal scheme to 
food security was seen in the context of other schemes running concurrently. Tamil Nadu provides a good 
role model -- the noon meal scheme runs in conjunction with the ICDS and the TN integrated nutrition 
programme.!® 


5 Dr. A.S. Seetharamu, professor of education at the Institute of Social & Economic Change, Bangalore, 
cited in Parikh and Yasmeen (2004). However, reports suggest that people have found a way round the 
scheme to divert money to their own coffers. See footnote 11, 

” See Parikh and Yasmeen (2004), 

19 Reports have come in from both Karnataka and Rajasthan, See also a study done by IIDS which 
documents how dalit cooks were opposed in a majority of the villages in Rajasthan and in a substantial 
proportion of villages in AP and TN (see Thorat and Lee, 2005). 

"| See Devraj (2004). 

'2 For example, in February 2003 the inspector of schools in Midnapore district was charged for the 
criminal offence of selling rice meant for the mid-day meal scheme in the open market -- in collusion with 
administrators and a private contractor (op. cit.). 

'3 The PIL which led to the Supreme Court order in 2001 making cooked midday meals mandatory in 
government schools is a notable example. Efforts have been made to draw attention to the functioning of 
government programmes providing some measure of food security to the poor — ICDS for the under sixes, 
the Public Distribution system, and midday meals for school children. 

'4 See the website of the Right to Food Campaign (www.righttofoodindia.org). 

'5 See Gopaldas’s note on the FRESH initiative. The FRESH initiative was launched at the World 
Education Forum in Dakar in 2000 to promote an integrated approach to school health, nutrition and 
education for the school-aged child. 

18 See Bennett (2003). 

'T These include ICDS, targeted PDS, old-age pension scheme, maternity scheme for BPL women, 
annapurna scheme, antyodaya anna yojana, family benefit scheme, and the midday meal scheme. 

'8 See Rajivan (2003). 


Benefits for enrolment and attendance 


The DFID study of impact of school-feeding projects 'S referred to above indicates that the evidence is 
strong on two counts: that the midday meal scheme impacts short-term hunger”° and that it has the ability to 
improve enrolment and attendance. This was also found by the Centre for Equity Studies field survey in 
rural Chhatisgarh, Rajasthan and Karnataka in 2003 2" Enrolment, particularly female enrolment had 
increased in response to the introduction of the cooked mid-day meal. Parents also reported that attendance 
had improved, because children were now keener to go to school. Numerous other studies conducted in 
rural areas in Bihar, West Bengal, Madhya Pradesh and Orissa have also found similar results?” These 
benefits need to be highlighted particularly in the context that the cooked midday meal scheme in India is 
not a programme which has been going on for very long nor does it have a powerful lobby in terms of its 
clientele of children from poor families. 


2.1 Focus of our research 


We felt a study of the midday meal scheme in Delhi would be useful, particularly to see the quality of 
public provisioning for the disadvantaged right in the capital itself. Several studies look at the functioning 
of the midday meal scheme in rural areas”? The midday meal scheme in Delhi, Bangalore and Hyderabad 
is different. Here the food is cooked in centralised kitchens while in rural areas and, for example, in 
Chennai, cooking areas are set up within the schoo! premises. So organisational requirements are quite 
different, but issues to do with quantity and quality of the meal served, as well as its potential impact on 
learning, on nutrition, on enrolment, and on attendance are similar. 


Media reports about the midday meal scheme are generally restricted to a crisis as for example when there 
is food poisoning and children fall ill.24 Even in such a situation the issue is politicised to embarrass the 
party in power rather than from any concern about what can be done to solve the problem. In fact, the 
extent of under-nutrition among children is little understood.?° NFHS-2 figures reveal that the situation is 
quite dismal for Delhi, though of course the extent of malnutrition was worst in the slum areas. 


Table 1: Health indicators for Delhi’s children based on NHFS-II data 


Proportion (%) of children aged 0-4 
ears who are underweight for age 

52.2 
33.0 


Underfive mortality rate (per 
1000) 
122 


Note: Underweight children are those who are more than 2SD units below the median of the reference 
weight for well-fed children recommended by WHO (1983) cited in Bhat and Zavier (1999). 
Source: Bhat and Zavier (1999). 


The focus of our research has been to probe into the current functioning of the cooked midday meal scheme 
in Delhi ~ right from when the grain leaves the godowns of the FCI to when it comes in its cooked form to 


'9 See Bennett (2003). 
20 A|though improvements in learning achievements are not so easily attributable to the school feeding 
rogramme (op.cit.). 
| See Dreze and Goyal (2003). 
22 See the website of the Right to Food Campaign (www.righttofoodindia.org) which has a very useful 
collection of articles and reports on the midday meal scheme. 
Op. cit. 
24 Th 2003 there were numerous examples of food poisoning in Delhi in particular in Wazirabad (see the 
Tribune, 29 Aug. 2003) and in Azadpur. In 2005 this was reported from a primary school in Karnataka. 
5 The problems associated with chronic malnutrition have received little attention. At the most, severe 
malnutrition in the context of extreme conditions such as drought in Rajasthan, or starvation deaths in 
Orissa is considered newsworthy. 


the school and is delivered to the children. We also wanted to use the research to suggest guidelines for a 
smoother implementation of the scheme. 


Primary education through government schools is provided by the Delhi government and by the MCD”° 
and by the NDMC”’, In our research we concentrate on MCD schools which cover the largest proportion of 
children enrolled in government primary schools, and which have the highest proportion of children from 
slum areas and disadvantaged socioeconomic backgrounds. 


Evolution of Delhi’s midday meal scheme in MCD primary schools 


Prior to 2001 school-children were being provided dry food such as bread, biscuits, murmure etc. and this 
continued till mid 2003.8 Under pressure from the Supreme Court ruling, MCD moved to looking at ways 
and means of providing cooked meals. Hyderabad and Bangalore were used as models. In January 2003, 
MCD invited Expressions of Interest from NGOs / caterers and others to supply cooked food to schools. 
The scheme was officially implemented on 1 July 2003 in only 410 MCD schools. In the next phase, 
starting from September 2003, more schools were brought under the scheme, and all schools were covered 
from April 2004. 


The financial allocation from the government was, and continues to be, Rs 2.40 per child: Rs 2 is given in 
cash by the local body (for pulses, vegetables, oil, condiments, fuel) and 0.40 paise is provided in terms of 
foodgrain (rice and wheat were to be lifted from FCI godowns in equal proportions) and paid for by the 
Ministry of HRD, Department of Elementary Education. In 2004-05, the allocations to cover conversion 
costs and transportation costs incurred by the suppliers is Rs 25 crores and the amount of foodgrain 
allocated is 90,561.40 quintals of rice and the same quantity of wheat2? 


MCD has decided to decentralise the entire system in view of the “gigantic task and the sensitivity of the 
matter”?° DEOs (District Education Officers) in charge of the different zones in Delhi have been given the 
responsibility of selecting NGOs and private caterers to do the actual cooking, handing over the foodgrain 
to these NGOs from the nearest FCI godowns, and reimbursing their expenses. 


MCD began with as many as 72 suppliers. This was whittled down to 56, then to 32 and now consists of 
only 11 NGOs, running 13 kitchens. Wilh this urrangement, cach supplier has a large responsibility 
(feeding from 25 thousand to 100 thousand children per day). MCD plans to have a five-year agreement 
with them, to be renewed annually. 


Regular evaluations through surprise visits to the kitchens and the schools by the Nutrition Foundation of 
India, an independent organisation, arc the primary means by which the cooking and the serving is 
monitored?! The Sri Ram Industrial Research Institute is also involved now in testing samples of food 
lifted from kitchens and schools. 


2.2 Methodology and sample 


The fieldwork*? had a school survey component and a household survey component conducted in mid 
2005. It was supplemented by visits to some of the kitchens where the food was cooked. 


26 Municipal Corporation of Delhi. 
27 New Delhi Municipal Corporation. 
28 An evaluation of the earlier scheme reveals that the food was irregular and that the afternoon schools in 
particular often received it in poor condition — packets torn, bread dry. See Government of NCT of Delhi 
(n.d.). 
ie See MCD provided “Note regarding midday meal in MCD schools”, prepared in 2005. 

Op. cit. 
3! The parameters for evaluation are based on guidelines of the Indian Standards Institution formulated in 
1972 and reaffirmed in 1995. 
3? Initial visits were made to Department of Education, MCD where we sought permission to do the school 
survey, and to Nutrition Foundation of India, whom we had heard was involved in the evaluation process. 


—_ 


In the school survey, observation was a key research tool as were semi structured interviews with the 
teachers as well as the supplier’s employees who were distributing the food. The school sample consisted 
of twelve schools in six outlying areas of Delhi where the proportion of slum population is high. In each of 
the six areas the morning and afternoon shift in the same school premises was done thus capturing the 
experiences of both boys and girls as the girls are generally allotted the morning shift and the boys the 
afternoon shift. This was also designed to make it possible to have a more homogeneous household survey 
with boys and girls from the household possibly attending the two schools being conducted in the same 
premises. The investigators spent 2.5 to 3 hours in each school to understand the pre-meal, meal and post- 
meal dynamics. 


During the household survey, we discussed parents’ and children’s perceptions of the cooked midday meal 
scheme as well as the infrastructure and the teaching quality in the school in which the child was enrolled. 
The household sample consisted of 10 households from each site, selected on the condition that they had a 
child enrolled in one of the sample schools in that area. 


3.1 Findings -- From the school survey 


A description of the event 


The meal arrived between half past eight and half past nine in the morning schools and between half past 
one and half past two in the evening schools. The food was cooked in centralised kitchens by separate 
organisations each serving a large number of schools. Usually it came in enormous aluminium containers in 
a Maruti van or truck, and was kept outside the Principal’s office until it was time for distribution. 


From the vantage-point of the schools themselves, the distribution system seemed to be working well. The 
meal arrived on time. There were no complaints of food getting spoilt because of time lags at any point 
between the cooking and the eating of the meal, although this could well happen in Delhi heat?* 
Preparation of food reportedly began 4-5 hours before it reached the school where it often sat for 30 
minutes to 1 hour before it was served. 


Serving the meal was a fairly smooth procedure. The distributor sent his own people, ranging from one to 
three persons to serve the meal. Children in class 5 sometimes helped the distributor with moving the 
containers to just outside the classes and in some cases with handing out the pooris or parathas. But for the 
most part they had only the welcome task of eating the meal. 


Generally, the children lined up outside the classroom and each child brought a tiffin -box’* into which food 
was ladled in from the container. The boxes were of different sizes. After getting the meal the children 
returned to the classroom in some schools and ate it there. In others the meal was eaten in recess on the 
steps, in the playground and in the classroom. 


Minimal impact on teaching time in the more functional schools 


The entire process of serving and eating was generally done within half an hour. However, the actual 
teaching time disrupted in each school varied with the general level of school functioning. Schools 
generally began the process of serving between 15-30 minutes before recess; the more functional schools 
began classes when the time for recess was over; the less functional ones allowed recess to continue well 
beyond the allotted time. 


33 ‘The school survey was done in April, when temperatures were soaring to 40 degrees C. In July and 
August, with high heat and humidity, there were even greater chances that the food could spoil. 
34 One school had distributed steel plates to all the children, 
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Quality and quantity 


In terms of taste, the quality of the meal was found to be sound for the most part with almost all items on 
the menu meeting the enthusiastic approval of the children. Many were observed enjoying every last grain 
on their plates, licking their fingers in delight. Rare was the child who did not take the food, and rarer still 
the school where the meal was not the highlight of the day. In particular, the children loved dal chawal, 
chhole chawal, and pooris. 


In a few schools the teachers did claim that there had been worms and other pests in the food in the past. 
Our observation suggested that it was not easy to monitor quality. Dal or subzi could be watered down, and 
was. The rajma was often seen to have a few beans of rajma and a lot of gravy. The vegetable pulao had 
no vegetables. It was the least popular item on the menu because in several cases the supplier was sending 
rice with only salt and haldi and little oil or vegetables to make it palatable and nutritious. Parathas had 
recently been taken off the menu because by the time they were served to the children they were too hard to 
eat. 


In terms of nutritive value, the food provided was supposed to satisfy a minimum in terms of calories and 
protein — teachers were informed that the meal will have 300 calories and 8-12 gms of protein. What this 
translates to in terms of quantity or the rationale for this quantity was not explained to the teachers. In 
practice the amount given to each child was no more than two ladles of food. It appeared to be less a meal 
than a snack. Secondly, the dal / chhole / rajma was often watered down— thus reducing the possibility of 
adequate protein content:> Third, the amount of food brought by the supplier was supposed to be based on 
the number of children who had attended the previous day2° There was no indication that there was such 
fine tuning between the school and the supplier. In spite of this, suppliers claimed there was rarely a 
shortage and even teachers said this. As discussed below, distributors had their way of stretching the food. 


From observation and from discussions with the children during the household survey, it appearcd that the 
amounts distributed varied. There was an incentive for the distributors to give the younger ones a little less 
since distribution started with the youngest and then went up to the next grade and so on. For one thing, it 
increased the possibility that there would not be less for the older children. It also increased the possibility 
that there would be some lett even after distributing the food to all the children. This in some schools meant 
second helpings for some children, and in others, a perk for the distributor.” 


There were also several cases when the food did run short when the teams visited, and there was little sign 
of extra supplies reaching there. 


Some attention paid to monitoring quality and quantity of food distributed 


In most cases teachers supervised the distribution of the meal and the efficiency of the system was greatly 
enhanced by this as also the fair treatment of each child2* In some schools the distributors functioned on 
their own without much teacher supervision. In at least one boys’ school, there was a lot of pushing and 
shoving in the queues to get the meal and the distributor commented that their work was more difficult in 
the boys’ schools, which could be because boys were less biddable than girls and because there was more 
teacher supervision in girls’ schools. 


The school had instructions that a number of specified adults would taste the food each day before it was 
served to the children?” This was not done or done at most in a nominal way. No signs of parent 


351 entils are themselves 30% protein and the rest carbohydrate. 

36 From a conversation with a supplier. 

37 In some schools, the supplier employed someone at the school itself for distribution; and these employees 
appeared to be allowed to keep the leftovers. 

3° This may not occur in as many schools as on days when there are no visitors. 

3° See MCD (2005a), p. 53. This had become particularly important when there had been several cases of 
food poisoning in 2003. 


involvement were seen. One HM reported that if a parent was visiting the school they did ask him/her to 
taste the food. Other teachers said that the suppliers were conscious that if the food didn’t pass muster it 
would be returned, and so they ensured that the food supplied was of acceptable quality. 


Lack of hygiene a major problem 


¢ While some of the distributors wore disposable gloves while serving the food, little emphasis seemed 
to be placed on their general cleanliness or their training. When the serving spoon , in one case, fell on 
the road as the container was being unloaded at the school, the spoon was simply replaced in the bag. 

¢ Inno school, did investigators see any cleaning up of the area where the food would be served or 
where the children would eat. In some schools, where children ate out in the open grounds, dust and 
dirt could well have found its way into the food. 

¢ What was also very serious was the fact that no one insisted that children wash their hands before the 
meal and only the rare child did. The importance of washing one’s hands with soap before eating was 
completely overlooked. Hygiene was a casualty even when there was adequate supply of water in the 
school, which was rare. 

e Children did wash their fingers and sometimes their tiffin-box after the meal. Even this was often done 
in an erratic way — in the corner of classrooms, in corridors without a drainage outlet. When the meal 
was eaten in classrooms it sometimes created a mess with dal and rice spilling on the ground.”° 


In some schools the poor hygiene was related to inadequate infrastructure. In one school for example where 
construction had not been completed the facilities for provision of water, for drinking und washing, were 
not yet in place. The new toilet block was also not functional. But in other schools, there were taps with 
running water, but not sufficient in number. In many schools, children brought water with them to drink. 


The situation with toilets was also poor. In nearly every school, there were dirty toilets. In some cases, the 
toilets were just locked up and children encouraged to go in the open. Sometimes this area could be just 
behind the classrooms where children ate their food. 


Opportunities to take advantage of supplementary benefits were not used as much as possible. In particular, 
little attention was paid to the socialisation value of the scheme. A good example of what is possible is 
provided by a researcher who visited an anganwadi in Tamilnadu, where children were observed to first sit 
together and wait until they had all been given their food and then say “thirukuaral” in unison before they 
began to eat.'' In one of the sample schools in Delhi, all the children had been given similar steel plates for 
their food, a step towards making the meal a common experience. Ultimately the teachers’ role in using the 
meal for object lessons in socialisation, hygiene, nutrition and so on is crucial, but there was little evidence 
of this. 


Teachers’ perceptions about the meal 


Teachers were generally positive about the meal. Since they were not involved in supervising the cooking 
or the distribution, most did not feel it was an additional work responsibility. Also they felt it was useful in 
their schools where there were many children from deprived socioeconomic backgrounds. 


But there were teachers who were negative for various reasons. 

e Several teachers preferred dry food because it was easier to check quantity or quality or nutritive value 
or it was easier to give it to the children, One teacher came up with an additional argument. Dry food 
had less chance of being confused by parents as a substitute for food they provided at home. 


4° This is so also because the size of boxes in which the children took their food differ so much. 
41 Prom an evolving draft of a Midday Meal Handbook prepared by Jean Dreze and S. Vivek. 
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e One teacher felt most of the children were not rice-eaters and would have preferred wheat-based food. 
One head teacher who was also a union leader was greatly against the scheme, He felt that there was 
little point providing a midday meal when the MCD was unable to provide proper drinking water in its 
schools. Although the situation with regard to drinking water is being given attention, we also found 
that this was still a problem. 

eA teacher felt that they had little power over suppliers because the suppliers complained to higher 
authorities and it was assumed that the teacher was corrupt and wanted a payoff. 


NFI evaluations on the functioning of the midday meal scheme in Delhi indicate a need for teachers to be 
sensitised to the need for such a programme.”? Whatever the efforts made by the MCD on this front, we 
also felt that more is required since the teachers’ role is crucial in the extent to which the midday meal 
programme is successful. 


3.2 Findings — from the suppliers 


Suppliers had varied backgrounds. Their names include words such as care, kalyan, sewa, manav vikas, 
stree shakti, clearly to indicate their identity as welfare organisations. But in general they were groups who 
treated this as a business opportunity, and there are indications that there are substantial profits to be made. 
MCD officials testified to the enormous growth experienced by these organisations as a result of their 
participation in the midday meal scheme. > The system was set up in such a way that these organisations 
needed to have deep pockets. The setting up expenses were theirs, and this could run into lakhs; also 
recurring costs were reimbursed with a time lag. 


With operations on a scale in which they are each providing meals from 25 thousand to one lakh children 
per day, the kitchens had to be highly mechanised — big boilers for steaming rice; large vessels for cooking 
vegetables; commercial gas ranges with 5-6 burners (connected to underground pipelines supplying 
cooking gas in one instance), chimneys above each gas burner in one kitchen; adequate lighting, exhaust 
fans. There were huge machines for kneading dough, for rolling out the dough, for cutting circular shapes 
to be hand-fried into pooris. In spite of the scale, one supplier visited was having pooris made by hand (by 
wamen — wamen were not generally visible doing the cooking of the midday meals elsewhere). 


he menu had been given to the suppliers by MCD, with the proviso that ley were not to repeat the same 
food in the morning and afternoon shift or on two successive days. This was a measure to ensure that 
leftover food was not served. The menus were largely cereal and lentil based. The suppliers found it easiest 
to supply rice (although one complained that FCI rice was much poorer quality than FCI wheat). The MCD 
suggested chawal rajma, chawal chhole, chawal dal, vegetable pulao, poori aalooand poori chhole. The 
items which were earlier on the menu and had become unpopular were daliya and parathas. 


Regular inspections by MCD and NFI meant that suppliers were used to their premises being visited. 

Nevertheless some suppliers were wary of the research team. What was visible in terms of organisation and 

cleanliness was mostly positive: 

e The locations of the kitchens were not in the vicinity of open drains or garbage dumps. 

e Waste management within the kitchens also appears to have been paid some attention. 

* Disposable gloves were being used in one kitchen (investigators were expected) for one or two 
operations. : 

e The kitchens had reasunably clean floors and walls and workspaces; in onc kitchen workers were not 
allowed to wear slippers. 

e Overall there was also no shortage of water reported or observed in any of the kitchens because tanks 
and motors had been installed. 

e The stores of grain, pulses, spices, oil were generally organised and kept neatly. 

But there were negatives too. Definitely disposable gloves were not used for all the operations even in the 

kitchens where some were being used. One kitchen still had a problem with flies. 


#2 The NFI study is not available in the public domain, as it has been done for MCD and submitted to them. 
Stn personal conversation with the research team. 
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There was not much evidence of fresh vegetables. Suppliers reported that they were procured afresh for 
each meal. The lack of visible fresh vegetables was in keeping with the menus which were mostly lentils 
and cereals. On the one day when the meal was poori sabzi, the sabzi was usually potato in a thin gravy. 


Suppliers were using mostly male staff for cooking and handling the machines. Much of the staff for 
cooking and distributing was employed through contractors: in this way the employer was guaranteed that 
the work would be done without having to negotiate wages and working conditions for the workers. 
Workers were in no position to unionise and make any demands of any kind. 


In general, Supreme Court suggestions about employing parents, women, lower castes to cook seemed to 
be given little importance. However, suppliers did indicate that they had been instructed to employ female 
staff as much as possible. If women were employed it was a few to clean cereals/lentils in one kitchen, and 
to clean the vessels in several kitchens. One supplier complained that he had more trouble with women 
employees -- they fought with each other and were always looking out for who had the lighter work. 


A very large number of staff were employed to distribute the food — between 100-150 staff going out in 
vans, tempos, and trucks. 


Suppliers complained that they had to function under many constraints. 

1. Heavy investment required in machinery: One supplier reported that he had invested between 7 to 10 
lakhs just on buying machines. Another supplier claimed to have invested crores. They were free to use 
their machinery for supplying meals elsewhere, as was done by some. 

2. Low financial allocations: Suppliers were getting Rs 2 per child. They felt those who expected them to 
provide better food were being unrealistic. 

3. Delayed reimbursement of expenses: The suppliers were supposed to be reimbursed after 2 weeks but the 
delays could go upto 3 months. They had to be able to access credit from the wholesalers themselves. 

4. Problems related to timely procurement of foodgrain: FCI foodgrain is erratic and insufficient. 

Private contractors supply them grain and pulses on credit. 

5. Problems related to quality of foodgrain: FCI foodgrain is of very poor quality, earlier kept for BPL 
card-holders. The rice becomes sticky when cooked. One supplier said that he pays Rs 2 per kilo just to 
have it cleaned and ready to usc. Others also reported that they exchange it for better grain in the open 
market at their cost. 

6. Lack of co-operation from teachers. This was reported by some of the suppliers. 

7. Constant visits to their kitchens which they felt constituted harassment. NFI came in for criticism. 
“Nutritionists are overly concerned with trivialities and know nothing about bulk processing of food,” said 
one supplier. 


From discussions with MCD and the teachers and the suppliers, it appeared that a great many of the earlier 
problems had been ironed out, and this was confirmed by looking through evaluations of the NFI from 
2003 onwards. The NFI indicated in its earlier evaluation that the state of the environment in which the 
kitchens were located were far from satisfactory — often the roads were katchha, garbage was strewn all 
over the place. In particular, suppliers often did not have good waste disposal systems within the kitchens 
themselves. Not enough attention was being made to supply nutritious and tasty food. All this was now 
vastly improved. 


3.3 Findings — from the household survey" 


From the household survey, it is clear that little attention has been paid by the MCD to let parents know 
about the purpose of introducing the cooked midday meal, or in including them in any way. In none of the 
60 households we visited in the 6 different sites spread over Delhi was any parent involved in cooking or 
distributing the meal (as part of a SHG or an NGO) or in monitoring the midday meal (by tasting it at the 


44 Some details about the respondents are available in the Appendix. In terms of economic status relative to 
other families in the slums, only 42% were poor or very poor. 


school) or in giving the authorities feedback on the content, quality or quantity of the food or even knew of 
other parents who had done so. 


In fact, the survey made it clear that parents had a lot of concerns which needed to be dealt with. 


1, Parents’ perceptions of the current midday meal scheme 


Just over half the parents in our sample did appreciate the quality of the cooked meal. They praised it 
because they felt the children were getting freshly-cooked and tasty food. “Cooked food is the best,” said 
the mother of a 11 year old boy, “but the quantity is insufficient.” Parents’ response to the quality of the 
midday meal was based mostly on their children’s opinion since they hadn’t tasted it as it was eaten in the 
school itself. 


Table 2. Parents’ response to quality and quantity of midday meal 


(per cent) 
Proportion of parents who said Quality of meal Quantity of meal 
they were: 

Satisfied 53.3 

Dissatisfied 45 

No response Te 


Note: In a few cases, the respondent was the child’s elder sibling or another relative living in the same 
house. 


Many children were also thrilled with the hot cooked meal. They found the meal very tasty and it ensured 
that they were in school at least for the first half of the day. Some made certain qualifications. A 10 year 
old girl in class 5 remarked, for example, that she enjoyed the meal except when the food has less salt or 
the rive was undercooked. Some parents were happy with the mcal because it made their children so much 
keener to go to school (this was said ahout hoth hoys and girls and also about children of all ages). 


However, as high a proportion of 45% of the parents in our sample were unhappy with the quality of the 

midday meal in general. The more drastic complaints included: 

© Cooked food is messy, unhygienic and a complete waste of time. 

e The food is not cooked properly. 

e The food is neither tasty or nutritious 

¢ The food has worms. Some also mentioned cockroaches and flies in the food. The food has stones and 
even glass. 


A few children reported that they found the food very unattractive and didn’t necessarily eat it. To some 
extent, they were indicating a caste and class bias to the large institutionalised feeding programme. They 
were fussy eaters and preferred to cat only home-food (9 year old Seema was from a better off family; ditto 
for 8 year old Rakesh who had recently been shifted here from a private schoo}). 


Some said that eating the food made them ill. These few children were confined to 2 sites. But clearly this 
is one of the most crucial aspects of running a successful midday meal programme — giving the children 
hygienically cooked food, and ensuring its consumption in a hygienic environment, as well as letting 
parents know that every effort is made to ensure this. 


Quantity issues: Some parents and children who were happy with the quality of the meal were unhappy 


about the quantity. And there were some unhappy with both. 
Their reasons for dissatisfaction: 
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e The food was not sufficient for growing children. 
e It was absurd to make children stand in such long lines for such little food. 
e Many felt the food was too little to make a difference to a child’s hunger. 


Some children made some additional comments related to quantity of food provided. 

¢  Acclass 5 student noted that in her school younger children were given less food. 

¢ Accass 5 boy noted that they were given very variable quantities since theirs was the last class to get 
the meal. 

¢ One distributor varied the quantity of food distributed according to the size of the tiffin-box. 

e Inone case, they were given extra rice if they wanted, but never extra pooris. 

e One child said the server hit them with the spoon if they asked for more. 


But again a little less than half of the parents and children did not have any complaints about the quantity of 

food being distributed. 

e Forsome the food was a substitute for what they might eat at home. Some of these children were small 
eaters anyway. This included younger children and some older girls. 

e For others, the food was a supplement to their regular intake. In several cases, after eating the food at 
school, the child came home during recess and ate what was available and then went back to school. 
The children who reported this pattern were generally boys. 


2. Majority of parents wanted the scheme to continue 


Notwithstanding the reservations about the quantity and quality of the food distributed in the midday meal 
scheme, around two thirds of those interviewed wanted the cooked midday meal scheme to continue. They 
were happy that their children enjoyed it though they didn’t necessarily think it was particularly important 
themsclves. Few were as enthusiastic as the 18 yr old elder brother who was the respondent in a household 
with 3 young children enrolled in primary school. He felt the meal was a great incentive for his siblings 
because it was tasty. Said another happy parent, “Children have become very interested in school after the 
midday meal started.” 


Table 3. Parents’ support for the cooked midday meal scheme 
Proportion (%) of parents who felt: 
Meal should be continued 


Although most parents wanted the scheme to continue, some qualifications need to be made. It was not that 
parents who were unhappy with the scheme were confined to families who were economically better off. 
Some “poor”? families also disliked the scheme but these “poor” families appear to come from 
“comfortable” rural backgrounds and expressing disaffection with school feeding programmes may reflect 

a class bias to the midday meal scheme in general.“* Such opinions were usually expressed by the better-off 
families whose views are given importance. 


3. Overall, parents did not feel that the cook a 

intake, or that it affected their eating patterns in any way. They felt that their children didn’t need the extra 
nutrition provided by the school meal, and in any case they didn’t need the government to provide any extra 
nutrition in school.4 Many “poor” parents did feel that perhaps the meal was useful for very poor families 
(from which they excluded themselves). 


43 Poor in terms of assets observed in the house. 

46 See Kumar Rana’s discussion of parents’ attitudes to the midday meal scheme in West Bengal. The more 
disadvantaged groups (SCs, STs, poor families in general were positive, the more affluent were generally 
opposed to the idea (Rana, 2004). 

47 As far as handouts went, parents were far more enthusiastic about the children being given books. 
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Apart from quantity and quality issues, cooked midday meals was also a new concept for many. Some of 
the parents were keen on a reversion to dry food being distributed as was done earlier. Parents suggested 
fruit, bread, biscuits, milk as options. The reasons were that it would be easier to deal with problems of 
quality and quantity control. Many children in one site had fallen ill so parents were particularly worried 
about the lack of hygiene. Dry food like fruit would also be more nutritious and healthy. Other reasons 
were also expressed — “Cooked food is a matter of taste,” said the mother of a little 7 year old girl; 
“Children like dry food they can snack on such as biscuits,” snid the mother of two children who prefer to 
come home to eat; “also there wil! be less time wasted in distribution.” 


Eating cooked food together by children of different castes was not articulated as a significant issue by 
parents. There were no reports of segregated distribution of food as has been reported in some rural areas in 
AP and TN during a 2003 study of caste discrimination in government food security programmes.'* 


4, Parents were not impressed with the midday meal as an incentive for regular attendance. They wanted 
more teaching, Although parents generally reported that the food was an attraction for the children 
themselves, and made them keener to go to school, they themselves did not think such an incentive was 
required, 


Overall, parents were very keen that the government should take some steps to improve the functioning of 
the school. Parents were keen that their children would learn something.” Raj in class 4 for example could 
not write a single word. Parents in one boys’ school complained bitterly about a teacher who beat his 
students in class 1; another who made the boys run errands for him; and a teacher who during the exarns, 
writes the answers on the board for children to copy. 


With the anxiety about a lack of regular teaching, parents were also upset that children were left 
unsupervised. In several schools, parents reported that this resulted in boys in particular getting into fights 
with each other, and in leaving school at will. Said the mother of 2 boys (9 and 10 years), “We don’t mind 

if they don’t give us anything but they should keep the children inside the school.” Deep down her fear is 
that perhaps the government is substituting providing education with a small cooked meal, and she doesn’t 
want it. 


About 40% of the parents were happy with the teaching quality in the school. Often they were happy with 
the discipline, or they were happy specifically with the teaching; in some cases it was a particular teacher; 
in others it was because they were in classes 4 and 5 and in some schools they were paid more attention. 


Infrastructural issues in the school: Parents and children complained bitterly about the lack of drinking 
water in the school. In some schools there was no drinking water at all. In one school, only one out of the 
five taps provided was functioning and there was always a big crowd around it. No water was one more 
reason for teachers to excuse children going home during the recess, and perhaps not returning to school. 


Access to clean and functional toilets would have been a chance for children to experience being part of an 
environment where cleanliness is given a high priority. However, little emphasis was placed on these 
niceties. Children reported that were told to use an open area in the school in all sites visited. Some children 
said that teachers feared that there may be snakes in the toilet blocks since they were used so little and so 
felt it was dangerous for little children to go in there by themselves. In some cases, toilets were functional 
and being used by children, but they were generally dirty. In other cases, toilets were functional but 
available for use only by the teachers. A need to use the toilet could be a sufficient reason for a child to 
leave the school during the school-day, and perhaps not return. 


All the schools we visited had boundary walls and gates which could be locked, but this was not done. The 
gates were generally open throughout the school day, and particularly in recess, it was expected that 
children might want to go home. More than half the households reported that children might forget to take 


‘8 See Thorat and Lee (2005). 
‘° Many mothers said that they send their children for tuition hoping that there they might learn something. 
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their tiffin box with them to school, in which case the majority would just return and get it. This is easy 
because the children live so close to the school, but it does imply that children have free entry and exit, not 
conducive to them attending a full school-day. In fact, 25% of the enrolled children reported that they 
regularly return early | from school (see Table) — certainly these were more likely to be boys °° or little ones 
six years and below,> 


Table 4, Frequency with which the child returns home early from school 
Proportion (%) of parents who said their child 
returns home early from school 

Regular! 

Rarely 


56.7 
100. 


[All 


In the next section, we draw the threads together from the different strands of our research and highlight the 
main issues. 


4. Discussion 


In contrast to the time when fruity bread and biscuits were distributed, the energy put in by the MCD into 

the functioning of the midday meal programme has seen a sea~ change.” In this context, one can observe 

that the Supreme Court’s injunctions since 2001 have been extremely important for Delhi’s primary school 
children, Today 9,50,000 children in 1863 schools** are being provided freshly cooked food on schooldays. 

The school survey revealed all the children enthusiastically queueing up und euling the food provided, so 
there is no doubt that the authorities are doing something right. It was hard to find a child who wasn’t 

eating. 


The MCD has also appointed an independent body like the NFI to conduct regular evaluations to ensure 
that the food provided is of acceptable quantity and quality. These evaluations are playing a crucial role. | 
NFI is using guidelines issued by the Indian Standards Institutiun>* On these parameters they are rating the | 
organisations providing the midday meals on an ongoing basis. It has resulted in a marked improvement in 
the quality of the midday meal scheme by ensuring better performance of the suppliers, The suppliers are 
conscious of the criteria involved in NFI’s rating (cleanliness of the cooking area, use of disposable gloves, 
cooks’ hair covered), although they are not necessarily sympathetic (prefer that evaluating agencies pick up 
samples of the food for testing). MCD was organising training for suppliers, but clearly improving 

standards of cleanliness and hygiene was not an issue which the suppliers felt that they needed training. 


Delhi’s suppliers did not seem to have nor were they being encouraged to have a broader vision. 

Although Delhi has looked for inspiration to both Hyderabad and Bangalore for successful implementation 

of the Supreme Court order, little attention has been paid to some of the finer details. Hyderabad for 

example is providing meals through an NGO (Naandi Foundation) which looks beyond its function of 

supplying meals. 

e It is conducting surveys in a sample of schools to evaluate the health and nutrition impact of the meals. 
This broader vision was not found in any of the organisations supplying meals in Delhi. 


5° In the case of boys, the possibility that they were attending a school in which there was little discipline or 
regular teaching was much higher. 
5''Tn the case of little ones, teachers seemed to pay them less attention than the older classes, and parents 
also seemed to give them more leeway to come home early. 
> In a study of the midday meal in Delhi in 1997-98, it was found that in the majority of schools, the meal 
was not even provided on a regular basis on all school-days. See the evaluation done by NCT Delhi for the 
period 1.4.97 to 31.3.98. 

These include 1819 MCD run schools and 44 MCD aided schools. Information was provided in a “Note 
regarding midday meal in MCD schools” by MCD, Department of Education. 
54 See Indian Standards Institution (1972, 1995). 


15 


e Even more significantly Naandi provides a feedback sheet to each school with the food cans. Even this 
type of mechanism could be subverted but it indicates that the organisation is willing to listen to 
feedback from the schools. 

Our survey also indicated that feedback from children and parents was crucial for the meal to have a better 

impact, whether it be through PTA meetings or otherwise. . 

e A sizeable proportion of children were happy with the food, and often wished they could get a little 
more. 

e Children were very enthusiastic about cerlain menus, and not about others. 

¢ Children showed their appreciation of changes made by the authorities when one supplier was replaced 
by another. They reported a considerable improvement in quantity and quality. 

¢ Boys in particular showed mixed eating patterns. Some were not eating the food, and went home to eat 
a proper meal during recess. At the same time others were eating the school meal, and going home to 
eat a proper meal during recess. 


There is considerable scope for improvement on several levels, and we shall discuss this 


with reference to Supreme Court orders as also with respect to benefits one could expect from putting a 
cooked midday meal scheme in place. 


1, To begin with, we shall highlight the deficiencies in terms of fulfilling of Supreme Court orders. 

a, Quality and quantity of meal served is likely to be below the minimum stipulated by the Supreme Court: 
Our research did not indicate that even this minimum goal of 300 calories and 8-12 gms of protein was 
being met. Our findings are based on observation of the food distributed in school and conversations with 
the children during the household survey. Many children mentioned that the quantities given varied (with 
the size of the children, the amount that was left, the size of the tiffin box), and that the quality varied (less 
or more beans in the rajma, for example, and lots of water in the sabzi). An evaluation done by the 
Nutrition Foundation of India in a sample of MCD schools confirms that in most schools children were 
getting less than the stipulated minimum in terms of calories and nutrients.°> 

b. Quality of rice provided by FCI was not of “good quality” as specified by the Supreme Court: 

The Supreme Court orders also specifically requires that the foodgrain made available by FCI be of good 
quality. However, there were complaints about the quality of rice from the suppliers— it was full of stones 
and it was impossible to cook in bulk, so they actually exchunged il in (he open market for a better quality 
of rice. The subsequent increase in costs was likely to be met by a drop in the amount spent on other 
ingredients going into the meal, impacting nutritive value and/or taste. 

c its. SCs and STs were not given preference in the appointment of cooks and helpers: This type of 
affirmative action did not appear to be of any consequence in MCD’s choice of suppliers or the workers in 
the kitchens.°® One organisation looked to employ women®! and one was also running women’s self-help 
groups but there was litle evidence of dalit women getting any increased milcage from this. 

d. Infrastructure to enable the scheme to function at optimum was not present in the school; While 
infrastructural facilities in the kitchens appear to have received considerable attention certainly facilities at 
the schools themselves were far from adequate. It is likely that there has been an improvement — there was 
evidence of construction activity, of new water tanks, but it was still not sufficient for children to be sent to 
wash their hands with soap before eating, or to wash their hands again after food, or to have water to drink. 


2. Potential benefits of the midday meal programme 


a. Impact on nutrition 


Is ita substitute or a supplement. Parents in our Delhi survey did not feel that the cooked midday meal 
affected their child’s food intake or eating patterns. In such a situation, the schoo! meal would be a 


55 The NFI study is not available in the public domain, as it has been done for MCD and submitted to them. 
Those supplying midday meals were not community-level organisations as mentioned before, and had 
little incentive to use these opportunities to improve the lot of those historically disadvantaged, even if 
there had been no clash with ideas of ritual pollution associated with SCs and STs. 

57 This group was employing widows of MCD employees. 
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supplement to the child’s normal intake and hence have some impact on their nutrition status and their 
overall health status. Of course the ext ent of the impact would depend very much on the extent and quality 
of the nutritional input, and the child’s health status overall. 


The meal needs to go beyond the minimum in terms of nutrients provided. A workshop on midday meals in 
July 2003 conducted by NFI in Delhi recommends that midday meals should seek to provide “for each 
school child, on each school day, roughly a third of the daily nutrient requirement in the form of a hot fresh 
cooked meal. This translates into an energy requirement of 350-500 kilocalories depending on the age of 
the child (and corresponds to about 100-125 gms of cereal). Apart from cereals, the meal should contain a 
good quantity of vegetables, particularly green, leafy vegetables, to combat deficiencies in micronutrients.” 
There was no attempt to meet this standard in the Delhi midday meal scheme. In particular Delhi’s menus 
pay scank regard to vegetables, specially green, leafy vegetables. At the most it is potatoes on 2 days in the 
week.*# 


Further, one should note that even this amount is far less than what is provided by the Andhra Pradesh and 
Tamil Nadu governments in their midday meal schemes. While Delhi’s schoolchildren are getting 300 
calories (at best) in keeping with the Supreme Court’s order which requires this to be a minimum, children 
in Hyderabad and Secunderabad get 365 calories. This is while Delhi’s budget allocation is Rs 2 per child 
and Hyderabad’s is Rs 1. 50.” In contrast to Delhi’s cereal and lentil based menus, Naandi Foundation 
provides rice, sambar and vegetables as a daily menu. 


In Tamil Nadu, the noon meal is a substitute for a home meal rather than a nutritional supplement. It 
comprises 100 gms of rice combined with 15 gms of dal with 1 gm of oil, 50 gms of vegetables, and 
necessary condiments together gi rotein at a cost of 1.29 per 
child per meal. 


The midday meul scheme needs tobe linked with other health inputs. Parents need nutrition and health 
education so they can be aware of the widespread undernouris hment among children in this age group, its 
impact and what can be done about it. Our survey indicated that without this, parents themselves saw the 
midday meal only as necessary for other poorer families, even when they themselves could be considered 
disadvantaged in terms of their relative standard of living. 


In addition, the child’s system has to be such as to be able to benefit from this dietary supplement. 
Several studies indicate that a nutritional supplement has limited value if given in isolation. 

The study of school-children in 3 districts in Gujarat found the children suffering from a host of 
infections,°° and strongly recommend that in addition to the meal, each child receives deworming tablets 
and iron and vilumin A supplements. 


Experts indicate that the combination of the midday meal with such health inputs could make a difference 
to the child’s learning capacity. 


Judging from the mixed reaction of teachers to the cooked midday meal scheme in Delhi (see section 2), it 
is clear that teachers also need to have much greater awareness of the rationale behind the cooked midday 
meal scheme and the potential impact on education of this and other inputs into the children’s health. The 
NFI has also stressed in its evaluations that all those involved in the implementation of the midday meal 
scheme need to be informed on various aspects for it to have the right impact. 


b. Supplementary benefits of the midday meal scheme were not being explored 


2 Nutrinuggets were mentioned but not observed. 
5° This rate applies to schools with over 100 children. See an article by Our Special Correspondent 
published i in The Hindu on 19 Feb. 2003. 

Intestinal worms, upper respiratory infections, malaria, diarrhoea, skin infections like scabies and many 
others were commonly found. 


17 


Ee 


— 


The major share of MCD’s attention appears to have gone into ensuring that the midday meal is cooked and 
brought to the school, and that there has been some control on the quality and quantity of the meal 
provided. More could be done on various levels. 
* Schools were neglecting the chance to teach the children something about hygienic eating habits, 
related both to personal hygiene and understanding the importance of the cleanliness of the surrounding 
areas. In fact the way in which the distribution of the meal was organised in schools was a powerful and 
negative demonstration of a lack of attention to issues of hygiene and overall cleanliness. 
© Children from one grade could be encuuraged to sit and eat together. In a limited way, the impact on 
socialisation among children was being realised to some extent. 
* Various aspects of a cooked meal could be made part of the curriculum. The meal gives teachers a 
chance to use a daily and shared experience in classroom teaching. For example, names of vegetables / 
colours / size / consistency for the younger ones; the nutritive value of different foods for the older ones. 
e Parents could be encouraged to have more contact and hence more communication with the school. 
e —Well-funded organisations could be encouraged to adopt schools and provide additional funds if 
required. 


ity and reducing the gender gap 


Improving equity. It is drawing children to school, particularly those from poor families. Enrolment has not 
increased greatly because numbers never enrolled at primary school tevel are already low.5! Teachers and 
parents indicate that attendance of children is much better and that the cooked meal is a draw. Our visits to 
the school suggested that the meal is a great highlight of the school day and children genuinely welcome it. 
In one school we observed children with eyes shut in finger licking bliss. 


10 year old Mithilesh from a poor family who is enrolled in class 4 says the midday meal makes 
him happy. It is hot. and he likes eating with his friends, His mother works a 12 hour shift ina 
nursing home nearby, while his 13 year old sister looks after the house. The cooked meal is a help 
for Mithilesh and his sister and mother since it feeds him and reduces the work they have to do. 


cing the vender gap, It has the potential to significantly improve the attentiveness of children who 
come from the 7.30 a.m. to 12.30 p.m. shift. The majority of these are girls and 55% (17 out of 30) of those 
interviewed said they came to school having had at most chai and fan (a long dry thin piece of toast). NFI 
documents suggest that breakfast is not part of the diet of most families in the lower income group’ ? Any 
contribution to nutritional levels of girls also has the potential to reduce the possibility of low birth-weight 
babies born to underfed mothers. 


Teaching time has not been reduced, It is important to note that the midday meal in itself is not disrupting 
the teaching environment in schools and that it has several beneficial impacts for children. Care has 
certainly been taken to see that teachers are not burdened by meal procedures. 


d. Lack of serious teaching and child security an independent and vexing problem 


Teaching quality was sorely indicted by 60% of households in the sample. So although children were being 
drawn into schoo! because many of the schools functioned poorly they were not necessarily staying on in 
school after the meal was distributed. 


Boys were particularly vulnerable to being in less functional schools, In 5 of 6 sites, the boy’s schools were 


strongly criticised by parents and children; only in 1 of the 6 sites was the girls’ school also strongly 
criticised. 


$1 See De et al. (2005), where household surveys in Delhi reveal that proportions never enrolled are very 
low and confined to girls from very poor families in particularly difficult circumstances. 


& See GOI (n.d.), Guidelines For Central Assistance Under The National Programme For Nutritional 
Support To Primary Education. 
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Several factors observed by the research team were particularly disturbing. 

e  Theclass 1 section and any nursery section were generally left completely to their own devices. The 
impact of such neglect on young children at such a sensitive time in their lives must be noted. Children 
who come away from the comfort of their homes for the first time need to be given very special care. 

e Inanumber of schools, although teacher pupil ratios from attendance registers were good, two 
sections were seated together. Sometimes there were over 100 children. 

e Insome schools, different grades were put together, with the younger children getting only passing 
attention. 

© Teaching activity, if any, was confined to a few children sitting in the front rows. 

Children were seen strolling in and out of the school gate, and not only during recess. However, during 
recess a substantial proportion of children were seen going home (in some cases to eat their regular 
meal). This does carry the inherent possibility that the child may not return to school. 


Parents were rightly disturbed by this situation and constantly reiterated that education was more important 
for them than any school feeding programme. What parents are asking for is what is already envisaged in 
plan documents where it is suggested that a Committee which should be set up to look after the 
implementation of the programme should ensure and look after the convergence of the midday meal 
programme with primary education. 


Children in vulnerable situations are most likely to drop out or be taken out as a result of poor functioning 
schools, since the perceived costs of sending them are high, and with low-quality schooling the perceived 
benefits of sending them are low.** This would negatively impact both equity and gender considerations in 
taking universalisation of education forward. 


5. Some guidelines for advocacy with parents and teachers 


We came across so many misconceptions and half-truths, that we felt parents and teachers would benefit 
trom having more information. 


1. A cooked midday meal is closely related to the children’s learning: its purpose is 

© to make school more attractive for children,®* and thus improve enrolment and attendance and reduce 
dropping out, 

e to improve attentiveness, memory and learning S* particularly in the context in which many children 
come to school on an empty stomach in the morning, Tand 

© loreduce the likelihood of children leaving school carly because of hunger. 


2. There ure ulher pusilive impacts of putting a cooked midday meal in place: 

e Children can learn about hygienic eating habits, related both to personal hygiene and understanding the 
importance of the cleanliness of the surrounding areas.°* Children are often more receptive to teachers. 

e Eating the same food in a society divided by caste and class is useful for the socialization of children 

e Children get a chance to learn healthy and nutritious eating habits.©? Children have a natural tendency 
to prefer snacks (mostly high in sugar, salt and / or oil) but these are generally less nutritious than a 
cooked meal. 


63 | inkages with other areas such as primary health care, early childhood care and education, nutrition and 
other related services have also been mentioned. 
64 See Deshmukh-Ranadive (2002) who discusses how perceptions of costs and benefits play an important 
part in schooling decisions. 

> Directly through providing food and through giving them an activity which they can look forward to. 
56 See Levinger (1996) cited in Ramachandran (2003). 
67 NFI (2003) makes the point that a good early morning breakfast is not part of the household routine. This 
was corroborated by our survey as discussed earlier. 

58 Such impacts are subject to very practical constraints such as lack of potable water in schools (see 
Bennett, 2003), very much the experience in Delhi schools we visited. 
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It can reduce children buying snacks from vendors. With such snacks, they don’t get value for money 
in terms of nutrition, and unhygienic preparation may lead to children falling ill and missing school. 
Irregular attendance is a common problem. 


. Why is dry food not a substitute for providing a hot cooked meal to children in the school itself. 
Its direct nutritional impact on the child is likely to be far lower compared to a cooked meal. The child 
takes the food home: bread or biscuits or fruit and may or may not eat it later. 
© = The child is attracted to a hot cooked meal, so he/she is more likely to come to school, and may be able 
to learn more (as discussed in the first para.). 


ew 


4. Parents could do with some input on nutrition, particularly as the UNICEF causal model of malnutrition 
highlights inadequate knowledge as a basic cause.” 
= What are the calorific value of foods and what constitutes a meal for a 5 or 6 year old; for a 7 or 8 
year old; for a9 or 10 year old. 
* How the ability to absorb nutrition is greatly reduced by frequent infections such as diarrhoea and 
respiratory infections. And that the vulnerability to these infections is greatly increased among the 
undernourished. 
= That the consequences of poor nutrition and health status is poor mental and physical growth. 
In fact, nutrition and health education is vital for parents, teachers and the community at large.”! Too much 
is taken for granted. 


For parents in particular 


e Parents need assurance that the government is concerned about the quality of the midday meal and has 
a scheme in place to ensure that the food is fresh and nutritious and tasty. Perhaps the government 
could prepare a simple list of do’s and don’t’s for suppliers, teachers and children which if shared with 
parents could put their minds at rest, quite apart from ensuring a better implementation of the midday 
meal scheme. 

* They need to be better informed about what the midday meal is supposed to provide. The quantity and 
content of the school meal is not enough to substitute for a meal. In fact, it is more like a snack. 


For teachers in particular 


Teachers play a crucial role in the success of the midday meal scheme. 

e They have to monitor the quantity and quality of the meal served to the children on an everyday basis. 

e They need to minimise its impact on teaching time 

e They need to maximise supplementary benefits such as socialising children of different backgrounds to 
eat together. 

e Atadeeper level, they need to be sensitised to be less patronising to children in their care. Such 
attitudes are particularly irksome when parents themselves feel that the children getting food reflects 
on perceptions about their poverty and inability to feed their children. 

© Most important of all, they need to be made aware of how parents and children are depending on them. 


6° NFI (2003) stresses how the midday meal could also be an opportunity to disseminate health and 
nutrition education to the parents and community at large. 

70\alamdana is a Chennai based organisation doing research on the knowledge gaps about nutrition among 
adolescents and young mothers in urban slums. See Nalamdana. 

7! Rajivan (2003) discusses these issues in detail. 
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Appendix 
School survey: 


The 12 government primary schools in the sample were not of similar size in terms of enrolment (see Table 
A.1). We had 1 site with small schools in north Delhi, 2 sites with medium sized schools in north west 

Delhi and north east Delhi, 3 sites with large schools in east Delhi, southeast Delhi and south Delhi. In each 
site, we visited the girls’ school and the boys’ school. All schools serve families living in slum areas and 
resettlement colonies, generally on the outskirts of Delhi. 


Table A.1: Enrolment in sample schools 


N NW NE 
Morning shift: Girls 284 405 446 
Afternoon shift: Boys | 181 472 421 


Note: N, NW,,... stand for North, Northwest, ... and give a general idea of the zone-wise 
location of the school. 


Household survey: 

There were 60 households surveyed with 6-10 year old children enrolled in the sample schools; 48 girls and 
56 boys — 104 in total. However, detailed information in each household was confined largely to one child 
in the household. 


The average age of the respondents was 37 years. The majority were female’? (80%). 


72 Not all the four-fifths of female respondents were the child’s mother. This group included 
grandmothers, aunts, and sisters. 
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Respondents generally needed the enrolled child to respond to questions about school functioning, and 
more particularly about the quantity and quality of the midday meal. In this the household survey benefited 
by being conducted in the holiday month of June. 


Most families were Hindu (86.7%), the rest were Muslim. In terms of economic status relative to other 
families in the area, close to one-fourth were comfortable, one-third were average, and nearly 42% were 
poor or very poor (see Table A.2). Families in our sample with girls enrolled in sample schools were 
generally better off than the families with boys enrolled in the sample schools: 55% of the enrolled boys in 
our sample were from poor and very poor households while the same figure was only 33% for the girls.” 


Table A.2 Economic status of households (per cent) 
Very poor Poor Average Comfortable | 
3.3 38.3 33.3 23'3 


Sources of livelihood among the men were generally confined to the unorganised sector. 
The range are given below. 
1.Employment 
¢ in private companies including a private bank, a company selling electronic goods: educational 
qualifications were class 10, 12 and graduation. Salaries in the region of Rs 3000 and above. 
e in shops and small establishments including book-binding, making cardboard boxes for sweets: 
nature of job not always clear but educational levels low and salaries as low as Rs 2000 per month. 
e _in factories: skilled labour. -- makes badminton corks; makes bartans; tailor; presswala in export 
factory (earnings about Rs 3000 per month) 
2.Self employed: 
e own textile business 
¢ in petty retail: corner shop selling groceries; own sweet shop; katran (strips of cloth) shop 
¢ — skilled labour: carpenter, electrician; presswala; tailor; driver; truck driver; painter, mistri; 
© itinerant providers of services: ply swings for children in the evenings; kabaadiwala; selling bangles on 
a cycle; sabziwala. 
3. Unskilled labour such as: 
e  Whitewashing / mistri 
¢ mazdoori (on construction sites) / beldari Rs 100 per day 


Sources of livelihood among the women: 

1, employed outside as domestics — cooking, cleaning; washing 

2, full time employment in institutions: ayah in a nursing home; tailor in a company; employed in a factory; 
in a private company 

3, home-based work included those: 

¢ Self employed -- making raakhees in family business; own katran shop 

e Doing job work — making bindis 

4. Mazdoori (Rs 40 per day) 


® Some sample households had both boys and girls enrolled in the sample schools. 
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Ahnexe re ~ 


RESEARCH: MID-DAY MEALS 


Introduction 


On 28 November 2001, the Supreme Court directed all State governments to 
introduce cooked mid-day meals in government and government-assisted primary 
schools within six months. The response of State governments was mixed: not all 
States were equally active in implementing these orders, and States which did so 
encountered significant "teething problems" along the way. However, early field 
teports also pointed out some major positive achievements of the new mid-day 
meal programmes, such as a dramatic increase in school participation in some 


states. 


Given the need for a balanced assessment of the state of mid-day meal schemes 
actoss the country, the Centre for Equity Studies (New Delhi) conducted a field 
survey of three states - Chattisgarh, Karnataka, and Rajasthan - between Decembet 
2002 and May 2003. The survey covered a random sample of 27 villages in each 
state, and involved detailed interviews with teachers, parents, cooks and others 


involved with the scheme. The research objectives were to 


e Evaluate the success of mid-day meal schemes with respect to the primary 
objectives of (a) hoosting school attendance, (b) improving child nutrition, 
(c) providing economic assistance to poor households, and (d) breaking 
caste barriers. 

° Investigate the problems faced by mid-day meal schemes, especially in 
relation to food quality, classroom disruption, and caste-based conflicts. 

° Search for "best practices" in the implementation of these schemes. 

¢ Understand how State Governments were able to finance mid-day meal 
schemes (this was particularly important since many State Governments 
claimed that such schemes were unaffordable). 

¢ Understand the role played by public action in persuading states to initiate 


these schemes. This would involve understanding where public pressure 


L 


came from, how it was built, and whether there is scope for similar 
coalition-building in other states. 
¢ Formulate concrete proposals for the introduction of mid-day meals in states 


where the Supreme Court order is yet to be implemented. 
Survey Questionnaires 
There ate six survey questionnaires: 


State: This questionnaire collects basic information on the coverage of mid-day 
meal schemes within the state, the different departments involved and their 
respective roles, allocations and offtake of grain, the costs involved, and how they 


are met. 


District: This questionnaire collects more detailed information on the extent and 
costs of mid-day meal schemes within the district, as well as information on how 
the district administration coordinates with administrations at the State and block 


level. 


Village Educatio: mmittee (VEC): VECs provide a way for local communities 
to be involved in the operation of schools. They ate intended to help communiues 
monitor the attendance of teachers and students, contribute services to the school, 
and encourage parents to send their childten to school and to keep them there. 
This questionnaire collects information on the constitution of the VEC, the role it 


plays in the mid-day meal scheme, and on whether the scheme has led to caste 


disputes. 


Teacher: This questionnaire collects information on how mid-day meal schemes are 
run at the school level, with details about the menu, the quality and quantity of 
grain supplied, logistics and infrastructure, and crucially, whether or not mid-day 
meals interrupt the classroom process. It also collects information on the effect of 


mid-day meals on school enrollment, absenteeism, and caste batriers. 


Cooks: This questionnaire collects information from cooks on food preparation 
and distribution. It includes questions about how the cook was appointed, as well 
as their caste, marital status, and salary. This information can be used to verify that 


State guidelines have been followed during the appointment process. 


Parents: Since mid-day meals are aimed at young children in primary schools, 


patents are likely to be a more reliable source of information on well the scheme 
works. This questionnaire asks patents whether they are satisfied with the quality of 
food and the infrastructure through which it is delivered, and whether they believe 
the scheme has made a difference to their children. It also elicits parental attitudes 


on matters relating to caste. 


There is an additional investigator questionnaire, used to record the investigator's 
overall assessment of how well the mid-day meal scheme is functioning, on the 


basis of observations made during the survey process. 


Mid-Day Meal Scheme Survey, 2003 
Guidelines 


1. General 


Background: The main purpose of this survey is to evaluate the 
Mid-day Meal scheme run by the government in Government 
and government aided schools. It is also an opportunity to learn 
about the economic and social circumstances of destitute 
households. This is not just an "academic" exercise; we are 
hoping that the results will help to make a case for a program 
with greater scope and better quality. 


Ethical aspects: It is important to conduct the interviews in an 
ethical manner, and in particular to avoid taking advantage of 
the position of power in which you may find yourself. For 
instance, make sure to introduce yourself, to explain the 
purpose of the survey, and to be considerate in accepting any 
refreshments that may be offered. Also, do not press the 
respondent for answers, when he or she is reluctant. 


Quality of the data: The success of this project depends 
crucially on the quality of the data. Your most important 
responsibility as investigators is to ensure that the data are as 
reliable and accurate as possible. 


Personal observations: We would like to associate you not 
only with the field survey but also with some of the discussions 
that will lead to the report. This means that we don't simply 
expect you to fill the questionnaires mechanically; we also look 
forward to learning from your personal observations and 
experiences. Even in the questionnaires, space has been 
provided for "personal observations"; please make liberal use 
of this opportunity to share your observations. 


What to do in a crisis: If there is a problem, you should call the 
Survey Coordinator and take action in consultation with 
him/her. Make sure not to interrupt the fieldwork, unless 
advised to do so by the Survey Coordinator. 


2. The questionnaires 


There are six qnestionnaires: (1) "Stale questionnaire", (2) 
"District level questionnaire", (3) "Village Education 
Committee questionnaire", (4) "[leadmaster questionnaire", 
(5) "Cooks Questionnaire", and (6) "Parents questionnaire". 
They should be filled roughly in that urder. In any case, you 
should slart with the "Village Education Committee 
questionnaire" and "Headmaster questionnaire", and then go 
on to the questionnaires for the Parents and cooks. 


3. Basic principles of interview 


e Normally, two investigators should work together 
on each interview. One investigator should conduct 
the interview; the other should record the answers 
on the relevant questionnaires. 


° Introducing yourself: The first step is to 
introduce yourself and to explain the purpose of the 
survey. You should also seek the respondent's 
consent for the interview. If he or she is unwilling, 
do not insist. 


If at all possible, avoid visiting households 
with the sarpanch or other local netas. If they 
follow you around, try to explain to them that you 
have strict instructions to visit the households 
without escort. There is no harm in getting help 
(e.g. for translation and related purposes) from local 
residents who don't have too much of a vested 
interest in this matter, e.g. a local teacher or social 
worker. 


If the respondent is a woman, try your best 
to get the answers directly from her rather than from 
male relatives, neighbors, etc. In such cases, it may 
be helpful to be accompanied by a local woman, 
e.g. the anganwadi worker or a member of the local 
mahila mandal. 


Probing: Some questions may not 
immediately elicit a response. In such cases the 
investigator will have to "probe" a bit. By this is 
meant that a few extra questions will have to be 
asked to make the question clearer to the 
respondent, or to get a more precise answer from 
the respondent. However the investigator must 
make sure that the meaning/sense of the question is 
not lost/altered in the process of probing. 


Try to avoid "leading questions", i.e. asking 
a question in a manner that encourages the 
respondent to answer in a particular way. We have 
tried to avoid leading questions in the 
questionnaires, but you will have to be aware of this 
potential danger also. Example: it is better to ask, 
"Do you feel that the Mid-day Meal Scheme is 
helpful or unhelpful?" instead of "Do you feel that 
the Mid-day Meal Scheme is helpful?” 


4, Filling the questionnaire 


Clarity is paramount. In particular, make sure that 
your handwriting is legible, and that all numerals 
are clear. Illegible information is useless. 


As far as possible, please do not leave any 
'blanks' in the questionnaire (blanks tend to create 
problems at the "coding" stage). Instead, use a dash 
(-') for 'not applicable’ and 'NR' for ‘no 
response’. The distinction between the two can be 
very important, please make sure to take it into 
account. 


If you find that an answer does not "fit" into the 
pre-specified format, you can record the answer in 
"long hand", as long as the information is clear and 
complete. Example: Suppose that in response to the 
question "Are you satisfied with the scheme? The 
only two responses are satisfied and dissatisfied and 
the actual response is that they are satisfied to the 
extent that the quality of food is good but 
dissatisfied to the extent that the quantity is too 
little. In such a case it would not be right to just tick 
on one of the two choices, it would be advisable to 
take a moment to write down such a response. 


In places where there is space for "qualitative 
information", please make an etfort to note any 
interesting details that may be relevant to this study 
(instead of "doing the minimum"). If you run out of 
space to note any details, please write at the back of 
the same page. 


Try to avoid "bogus answers". If a response strikes 
you as non-credible, it's better to write "NR" than to 
record a bogus answer. 


Use the relevant spaces to record the answer, 
especially the quantitative answers (there is a 
clearly demarcated "box" for each numerical 
answer). Try to avoid writing in margins, but if you 
do, use the left-hand margin, not the right-hand 


(1) 
(2) 
(3) 
(4) 
(5) 
(6) 


margin (using the right-hand margin may confuse 
the data-entry operator). 


Please make sure that all questionnaires are safely 
stapled. Loose pages are a headache since one may 
not know which questionnaire they belong to. 


Use Arabic numerals (1,2, ctc.), not Roman (I, 
Il,...), Hindi, ete. 


In case of questions that have multiple answer 
choices let the respondent answer the question on 
his/her own and you must record depending on the 
relevant answer choices. If you ask the different 
choices directly you may end up leading the 
respondent. Sometimes when the respondent does 
not understand the question then the answer choices 
may be used as cues. 


In cases where you have to choose between 
different options, and each option has a "code" 
attached to it, please circle the relevant option, 
including the "code". 


Example: Q 15 from Part 6: If dissatisfied, what do 
you feel are the main reasons for low quality of the 
food? (Tick more than one response if necessary) 

Inappropriate menu 

Grain is of low quality 

Ingredients are of low quality 

Absence of a competent cook at school 

The cook is careless 

Other reasons (please specify) 
And the responses are the first two then make sure 
to circle the relevant codes as well (1) and (2) in 
this case. 
Also remember to record these responses in the 
extreme right column. 


uestion-wise clarifications and comments 


Part 4: Headmaster Questionnaire: 


e While questioning the Headmaster it is important for you to 


understand the difference between enrollment and attendance. 
Enrollment is the number of students that have enrolled for 
admission to the school, i.e., the students whose names appear 
on the attendance register. Attendance on the other hand is the 
number of students actually attending school, i.e., present on a 
particular day. Data on enrollment and attendance figures is 
crucial for the purpose of our survey and you must take utmost 
care while recording these figures. Getting figures for 
enrollment would be fairly simple, this you can get from the 
school attendance/enrollment register for the year 2001 (last 
year) and 2002 (this year). Getting figures for attendance 
would be a more challenging job. We want you to record the 
total attendance for each day of the month preceding the 
survey, for all students of class I and for classes I-V taken 
together. We also want the same figures for the SC/ST children 
in class I and for classes I-V taken together. For Sundays and 
other holidays put a cross “X” in the relevant cells and for days 
when the data is missing put “M” in the relevant cells. And 
please fill all the cells and blanks that need to be filled out 
(empty cells can make the data unusable for us). 

A “double meeting” system is one in which attendance is 
taken twice, once in the morning and once after lunch break. In 
these schools the attendance figure may appear conspicuously 
magnified. If the school you visit has a double meeting system, 
just record the figures as they are and mention that they are 
double meeting figures. 


Qn 7, here we want to know the number of classrooms only. 


Qn 34 and 36, if the headmaster in unable to calculate the 
percentage figures you try to do it using the figures of last 
year’s enrollment (LE) in the given month and the current 
year’s enrollment (CE) in the same month by using the 
following formula: 


CE-LE x 100 
LE 


Qn. 37 and 38, please read the question carefully, here we are 
talking about absenteeism and not the attendance. Make sure to 
explain the difference between the two. The question 37 reads 
“ In your perception, has there been any change in pupil 


absenteeism this year compared with last year?” If the response 
is yes, then you go on to the next question which concerns the 
direction of change. The given coded answers are “increased”, 
“decreased” and “no response”. If the headmaster says that it 
has increased then the implied meaning is that attendance has 
gone down, so double check with him/her to make sure they 
have understood the question properly. 


Qn 44, this question reads, “Have some upper-caste parents 
ever objected to their children sharing a meal with children of 
other castes?” If the response for this question is “No” then Qn 
45 and Qn 46 (parts 1,2 ,3 & 4) are not applicable, so you must 
put a dash for each of these questions. 


Qn 49, this question is about the water arrangement if 

> the response is that water is brought to school 
from the nearest source (code 2)then Qn 
50(partl) becomes not applicable as that asks 
about the source of water within the school 
itself. So put a dash for Qn50.1 and go on to 
Qn50.2. 

> the response is that there is a source of 
drinking water at the school itself(codel), 
then Qn 50(partl) is applicable but Qn 
50(part2) becomes not applicable. 

> the response is that children go to the nearest 
source to drink water, then Qn 50 part 1 and 
2 both are not applicable, so put dashes for 
them and go to Qn 50 part 3. 


Part 5: Cooks Questionnaire: 


Qn 14 and Qn 15, through these questions we want to know 
how the fuel is arranged for the MDM. The response to Qn 14 
will be “yes” only if fuel as such is provided to the cook. If the 
cook is given money for the fuel then the response to Qn 14 
would be “No” and the payment for fuel should be recorded in 
Qn 15. 


Qn 17 try to elicit as much as you can regarding the MDM. 
Probe into how the cook actually cooks the food starting from 
cleaning the grain to giving the cooked meal to the school. 


Qn 30 from this response try to gather who and what may have 
been the main reasons for his/her selection and then verify in 
later questions. 


Qn 33, this is about the cook’s remuneration. Please note that 
this can be in monetary figure or in kind. Whatever it is please 
note the monthly remuneration. 


Part 6: Parents Questionnaire: 


Sampling: You should select the parents according to the 

following criterion: 
We will be visiting three villages in a particular 
block. Take parents of class I children in the first 
village, of class II children in the second village 
and class V children in the third village of each 
block. Take the attendance register of the relevant 
class and take the first SC/ST girl and then the first 
General irl, then the first SC/ST boy and then the 
first General boy. (follow this in all the blocks). 
This way you will have a sample of four parents in 
each village. 


Qn 7 if the respondent does not know the amount in acres then 
write in whatever unit they know and get the local conversion 
into acres. 


Qn 8 this question is based on your observation of the village 
as a whole. So be observant and record what you feel is 
appropriate. 


Qn 10,15,25 while asking this question observe how they have 
been phrased. We have tried not to make it a leading question, 
so while asking make sure you preserve the unbiasedness of the 
question. 


Findings 


The findings reported below are taken from an article by Jean Deze and Aparajita 


Goyal which is available here. A preliminary report by Bhanupriya Rao on the 


Kumbalgath tehsil, Rajasthan, is available here. 


The survey at a glance 


e In 76 of the 81 sample schools, investigators found that mid-day meals were 
being served regularly. 
e Infrastructural facilities (cooking shed, water supply, etc.) vary widely, and are 


often inadequate. 
School cnrolment goes up 


e Class I enrolment increased by 15% in the three States combined, between 
the 2001-02 and 2002-03 academic years. 


¢ Female enrolment in Rajasthan incteased by 29% during this period. 
End of classroom hunget 


e Since children often come to school without any breakfast, they often find it 
difficult to concentrate on an empty stomach. Mid-day meals seem to have 
resolved this problem. 

° Mid-day meals have also helped to avert an intensification of child 


undernutrition in many drought-affected areas. 
Overcoming caste. barriers 


e The experience of sharing a meal together may help overcome caste barriers. 
‘The survey found little evidence of open discrimination; most lowet-caste 
parents did not feel that their children had been subject to discrimination. 

e However, there has been upper-caste tesistance to the appointment of Dalit 


cooks. 
The way forward 


¢ Financial allocations need to be raised: Rajasthan spends only Rs. 0.50 pet 
child per day on the scheme at present. 


e Mid-day meal infrastructure needs to be improved. 


The monitoring system needs to be ovethauled. 
e Caste issues need to be addressed. 


e Lunch menus need to be varied and made mote nutritious. 


« Mid-day meal schemes could be linked to broader health initiatives. 
e States such as Bihar and Uttar Pradesh must be persuaded to implement mid- 


day meal schemes as soon as possible. 


http:/www.righttofoodindia.org/research/mdm/res 
mdm _findings.html 


http://www. flonnet.com/f12016/stories/2003 081500 
2208500.htm 
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